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ABSTRACT 
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FOREWORD 



Corrections educators have become increasingly aware of the need to obtain Credible 
information for accountability and improvement of vocational education programs. Vocational 
education is often viewed as an intervention as it provides inmates with occupational skills that 
will hopefully deter their returh to prison. However, th^re are other outcomes that are often used 
in the evaluation of these programs such as employability skills, career development, and self- 
esteem. Regardless of the outcomes selected, corrections educators need information for 
improving the quality of their vocational education programs. 

In ordej to provide this information, corrections educators neecj to understand evaluation 
procedures. Corrections educators have expressed the need fof an evaluation guide that 
specifically addresses the need, processes, and outcomes of vocational education programs < 
offered in correctional settings. Developed in response to this need, this handbook is a guide for 
improving current procedures or initiating procedures to evaluate vocational education programs 
in cQrrectional institutions. 

' 'The handbook guides the users' introduction to and understanding of the evaluation process 
through the use of a case study, flowcharts, and checklists. Descriptions of helpful resources and 
examples of instruments provide additional help for the corrections educator charged with the 
responsibility to evaluate the vocational education program. 

0 

" m i 

■ 'Several nationally recognized practitioners and experts in correctional education provided 
valuable insights and information for the development of this handbook. These individual 
included Alfons Maresh, Minnesota State Department of Corrections; hfartzel Black, Vienna * 
(Illinois) Correctional Center; Rex Zent Ohio Department of Rehabilitation and Corrections; A. 
Neil Johnson, Ohio Department of Education; and Joseph Rowan, Executive Director of the 
American Association of Correcflbnal Officers. 
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EXECUTIVE SUMMARY 



) 



Recej*it]y, Supreme Court Chief Justice Warren Burger strorigly urged . that inmates be given 
opportunities to leave prison with marketable skills. Vocational education is viewed as critical for 
\ providing inmates' with a chance to obtain and maintain jobs upon release. Relevant, effective, 
and efficient vocational education programs are essential to providelnmates with these 
opportunities. 

The understanding of Evaluation procedures* is necessary in order to obtain credible 
information for improvement and accountability of vocational education programs in correctional 
institutions. Therefore, there is a nefedsfor an evalu9tion guide for use in vocational education 
programs offered by correctional institutions. / 

. . ■ . ~* * . " 

The handbook is designed for corrections educators with little or no experienced * 
evaluation. In the fUst section evaluation is defined, while in the next section vocational 
education is differentiated from pre-vocational training or work in prison industries. The next 
* efght sections describe the evaluation process: Determining the Purposes-of Evakiation; 

Identifying Outcomes; Determining Questions; Selecting Methods; Identifying Staff; Collecting , 
Data; Analyzing Data; and Repprting the Results of Evaluations. 

The handbook includes checklists, worksheets, and a hypothetical case study to guide the 
user through the evaluation process. .Examples of evaluation materials from various correctional 
and educational agencies as well as annotated recommended resources are&lso provided. * 
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PART I 

Evaluation of Vocational Programs 



The increasing emphasis on evaluating vocatiortaf education* programs in correctional 
institutions has created a need for dn evaluation guide for corrections educators. Since the late' 
1960s there has been a growing emphasis on evaluating federally, funded Social and education 
programs. Concern for evaluation of vocational education programs has greatly increased since, 
passage of the 1976 Education Amendments. These Amendments stipulate that federal fupds for 
vocational education are contingent upon evaluation for program improvement. In ordeijto- 
measure the-effectiveness, efficiency, and relevance of their programs, corrections educafors 
|F must understand evaluation procedures. 

Recently the vocational education of inmltes has been viewed as critical to the reduction of 
recidivism.* in his address to the American Bar Association in February 1981 Supreme Court Chief 
Justice Warren Burger linked the lack of education and marketable skills to criminal behavior. He 
urged that jnmates Be given the opportunity to change through education and vocational \ 
, training. Burger (1981) said tt\at: ^ 

We must accept the reality that to confine offenders behind walls without trying to 
change them is an expensive folly with short term benefits— winning of battles 
while losing the war. , * 

> 

Burger proposed that all vocational and educational programs be made mandatory, with 
credit against sentence for educational progress. He explained that it would be, "literally a 
program to learn the way out of prison," so that no prisoner leaves without being able to read, 
wtite, do basic arithmetic and demonstrate a marketable skill. According to Carlson, U.S. 
department of Education, "Incarceration has dramatically increased durir^g the last decade." 
There'cffrently are 446,000 inmates in 9I2 state prisons, 4,000 local jails, and\49 federal prisons. 
Most inmates, 95 percent, will eventually be released\Approximately 150,000 inmates are .released 
each year. In the federal prisons about half of the inmates have access to some type of 
education. The majority of inmates are in state prisons\where about a tenth are able to acquire 
basic or vocational skills. \ 

• • 

The average cost for keeping one adult inmate in prison for one year is about $13,000, not 
counting the court costs, welfare, payments to families, and numerous other costs associated 
with crime. Jn terms of cost to society in dollars alone, it makes sense to provide adequate 
vocationaMraining to give former inmates a chance to do legitimate work. A" 1979 Government 
Accounting Office^ report pointed out th&t correctipnal institutions can do mor6 to improve the 
employability of former inmates. 1n this society, regular employment is ttje^accepted way of , 
assuming responsibility for oneself. £ • ' . Kj * 



Employment not only affects the former inmate's ability to be self supporting without 
recourse to crime but also influences use of leisure time, associatiop with friends, self-concept, 
and expectatiorifcfor the future. Carlson reported that at the August 193l'National Advisory 
Council for Vocational Education hearings, former innr&tes testified that, if they .had not had 
vocational education classes while in prison and been given job opportunities when released * 
they would "very likely have returned to prison*" 



Unique Setting for Vocational 1 Education Programs 

Correctional institutions have always faced the problem of what to do with inmates. It is 
generally agreed that corrections should not only maintain custody,, but also achieve some 
degree of rehabilitation. According to Al Maresh, Minnesota Department of Corrections, 

9 i 

These two goals, xustodyjand rehabilitation, must be accomplished using sound • 
* management and good performance accountability. These goals are often 
contrary and therefore difficult to accomplish. , 

When considering the ways to constructively occupy inmates' time, the few alternatives 
include prison industries, education, vocational training, and counseling. Most inmates have |pv 
levels of schooling and v^ork skills. The typical inmate is male (96 percent), white (51 percent) or 



black (47 peroent), twenty-three years old (53 percent are twenty to thirty yeapS old),has had ten 
years of school (75 percent below, hig^h school level), eAcned less than $10,000 per y£ar (80 , 
percent), and never was in the military (73 percent) (Carfson 1981). 



The»extrem£* variability among the inmate populations^With rSspect to formal education, work 
experience, specific skills, motivation, varying lengths of sentences, an<J other factors unique to 
each institution makes the task of providing suitable vocational education quite formidable. 

* In contrast to schools in the free wprld, Maresh describes vocationafeducatlon in a 
correctional institution as providing instruction in a setting constantly bombarded with security 
aeeds, maintenance needs, treatment modes, and competing activities. These activities limit the 
Hime available for vocational education instruction. Furthermore, educators and educational 
administrators are typically isolated from other parts of the correctionaWffstitution. Often there 
are misunderstandings with custodial and other staff members who may have difficulty 
understanding the educators' need to^build trusting relationships with inmates in order to \ ' 
enhance their learning and skill development Currently there^re extreme variations among the 
states in mandating and supporting vocational education programs in correctional institutions. 

Since the 1960s th.e emphasis in corregtional education has been on vocational training. , 
Resources for vocational- education programs are frequently minimal, often depending upon 
contributions from local industries and the dedication of staff to maintain operation. Many adult 
correctional institutions are Old, were designed as inrtdustrial prisons, and lack proper space,and 
equipment for vocational education programs. 



Purpose of This Handbook 



CdrrectiQns educators know that quality vocational education can be provided in prisons. 
6orrections educators also know that inmates benefit^om the participation in vocational 
education programs. Although causal relationships have been difficult to determine, correction? 
educators believe ttfat vocational education has rehabilitative value. Despite these beliefs, there 
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have been no outstanding studies or evaluations to indicate the role voqational education plays 
in rehabilitation and reduction of recidivism of inmates. ' / 

* / . V \. 

A guide for the evaluation of vocational educartion programs in correctional institutions is 
necessary because there isr no evaluation material specifically developed for use by corrections 
educators. Many of the criteria that havj been used to evaluatejhese programs are not reafistig 
because the progfams are usually based upon a public scfiool setting that is not alw&ys 
applicable to correctional mstitutions* Standard evaluation texts aqd ''how-to" manuals seldom J 
cover evfctuatibn of education systems in correctional institutions. X - 

Corrections, educators 'Reed, a guide for evaluating their vocational education programs in the 
context of the uniqueness of correctional institutions and the corqplex roles played by the 
intaates, staff, and administrators. Maresh points ouUhat historically, corrections adminisfcators 
have had considerable local autonomy often causing them to view evaluations With suspicion. In 
reality, almost everyone views evaluation with some suspicion or apprehension. Educators can 
promote the evaluation .activities by demonstrating that evaluation will help improve their 
^programs by providing information for planning, allocating respurces, and demonstrating, 
accountability for the resources previously usee}. Through the evaluatioaof corrections programs 
educators can provide information that vocational education" is a vfable intervention and jl 
beneficial alternative from among those available to inmates. 

s * * 

Joseph R. Rowan, Executive Director of the Americartl Association of Correctional Officers 

and also the Executive Director, Juvenile and Criminal Justice International, Inc., cautions that. 

Experience has demonstrated that the ag#fcy which evaluates itself or asks an - 
outside consultant tQ conduct tha evaluation is in a much better position to 
resolve the problems which are found than the agency which goes through an 
- evaluation due \o a request or pressure from the outside. Then, the agency is 
defending itself and is reactive rather than proactive. 
A " , 

How to Use This Handbook 

i 

This handbook was designed ffir coy-ections educators who have had veryjittle or no^ 
experience with evaluation. In this handbook, the process of evaluation is presented in eight 
steps as shpwn in figure 1. 

In this handbook, each step of evaluation is described briefly in relation to the correctional 
setting. The Suggested approalWlg that yoy read the handbook through for a general ^ 
understanding. Then, start at the beginning and complete the checklists and worksheets. There 
are no right or wrong answers. 

( .A 

The checklists, indicated by a V ,help you think atfcut the evaluation step that is being 
discussed. Use the. checklists and worksheets to guide you in planniog the steps of your 
evaluations. 

At the beginning of most sections in the handbook there is part of a case study about the 
evaluation of the vocational education program at Smithville Correctional Institution. This 
hypothetical cise study describes the evaluation process in a correctional setting. Relate the 
case study to evaluating the Vocational education program in your institution. 
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FIGURE 1 

FLOWCHART OF THE EVALUATION PROQESS 
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The sample materials section contains several useful instruments and letters^hich were 
adapted from ones used by vocational educators. You may find the checklist whjcn^as adapted 
from the American CorrectionaLAssociation standards for vocaflonal educatior/esp^ially useful. 
There are several other helpful materials about evaluation described in the resources section. 
You may find some of these in a library, others can be ordered at nominal cosf 
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^ ■ defining evaluation 
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One definition of evaluation of vocational education programs in correctional institutions is 
f that: ' % ' 

Evaluation is a way to compare questions about the outcomes we hoped would 
occflr from the vocational education program with results that actually occurred 
because of inmates participation in the vocational education program. 

Evaluation b^ins ^ith identifying the desired outcomes of vocational education in a 
correctional institution. The procedures of evaluation* are outlined in eight steps in this 
handbook: (1) determine purposes, (2) identify outcomes, (3) determine questions, (4) select 
methods, (5) identify staff, (6) collect data,^) analyze data, and (8) report results By no means 
are these the only steps'that can be followed in conducting evaluations. These eight steps, 
however, will lead you through the process of planning and conducting evaluations for outcomes 
in a logical, systematic way. This Jype of evaluation is often called product evaluation. ^ 

Another type of evaluation is process evaluation. Process evaluation is concerned With J 
looking at the processes in vocatienal education that influence the outcomes. There are mar>y i 
cesses,. including the usefulness or adequacy of: v ' r* r * 

facilities? equipment, and instruction % 

^fctirriculum m r , ' ~ 

• program planning procedure^ ' ^ 

• admissions criteria ^ 

♦ 

• staff qualifications 

• performance requirements 

.This handbook provides examples (Example section) that are useful for process evaluation. 
The checklists adapte.d from the American Correctional Association and the "Minnesota Guide 
for Self Evaluation" list criteria for process evaluation. 
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The definitions of vocationafeducation in correctional institutions vary considerably. 
Vocational education programs are'sometimes confused with prevocational training or prison 
industries. The National Advisory CfcunciLon Vocational Education (NACVE, 1981) uses the 
following definition of vocational education: 

Vocational education in cqr/ectrons can be broadly defined as instruction offSied 
within correctional systems to enable offenders to be employment ready upon 
• their return to free society. It involv.es the development of basic skills, specific 
occupational training, and art array of "job readiness" training, including the 
development of motivation, .good work- habits, and survival skills. 

^Vocational education pToqrarr^ offered in correctional institutions vary across the states and 
within states. Some of the variations are due to the types of inmates. For example, institutions for 
juvenires tend to offer career education, prevocational training, or job exploration. Programs for 
adults tend to offerskill tratnftig in specialized job are^s. Programs differ in institutions for 
female or male inmates and forth© type of security— maximum, medium, or minimum- 
maintained. \ * 

A^recenf report by Carlson, U.S. Department of Education, fJtovided a description of current 
correctional vocational edacation programs and Inmates. The larger state institutions offer an 
average of ten programs whereas the smaller ones offer four. The most cpmmon skill training 
offered is for trade and industrial jObs. Typical vocational education courses are auto mechanics,, 
masonry, carpentry, electrical wiring, plumbingf welding, machine trades, radio and television 
fepair, and small engine repairt. Other, less common bourses are food services, building ( 
maintenance, grounds keeping, j>artending, 'shoe repair, upholstery, home economics, 
cosmetology, business and office/education, and health occupations. 

'Although most of the vocational education programs are self-contained, some correctional 
institutions' contract with outsi.de educational institutions to provide the courses within tffe 
institution Or at an outside location. Some correctional institutions offer a combination Of self- 
contained and contracted courses. • ■ . ' 
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Use checklist^/ to decrde if your program meets the definition of a vocational educatio^ 
. program accepted by the National Advisory Council on Vocational. Education. 



Determining if You Have a Vocational Education Program 



Please check yes, no, or not sure. 



1. Will the program enable inmates to be reatly for employment 
upon their return to society? 

, c • 

2. Does the program develop inmates' — * 

• Basic ^tills (reading, writing, math, etc.)? 

• Occupational skills for a specific job or cluster of jobs? 

• Job readiness, in terms of motivation, wo^k habits, and 
interpersonal relationships with employers and coworkers? 

> 

• Job search skids (interviewing, filling out employment 
applications, preparing a job history sheet, etc.)? 




- NOT 
YES NO SURE 



□ □ □ 



□ □ a 
pan 

□ an 



3. Is there an advisory committee with representation from the* 
free world (business, industry, labor, and education)' to establish 
standards for program content and program evaluation? 



4. Is the prpgram established on the basis of documented information 
showing the needs of state and/or local employers for certain 
occupational skills (needs assessment)? 4 



□ □ * □ 



If a progranj is to meet the National Advisory Council's definition of a vocational education 
program, the answers for the first ar>d secon^ questions should be yes. If no, your program 
either provides pre-vocatiarfal trainmg or is essentially a prison industry, not a vocational 
education prpgram. Ideally, the third and fourth qdestions would also be answered yes, but the 
realistictime constraints upon corrections educators often preclude needs assessments and ^ 
actively functioning advisory committees. 4 
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- In many cases the press tq develop evaluation procedures and to evaluate the vocational 
education programs will be the reason for startjngand maintaining*an advisory committee ' ' 
Similarly, needs assessments are frequently^developed jas a part of the evaluation procedur^f to 
prpvide realistic information about the skill needs of employers in the vicinity or across the state 
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EVALUATING THE VOCATIONAL EDUCATION 
PROGRAM AT SMITHVILLE CORRECTIONAL INSTITUTION 

• ... ^ 

Pat Howell has been the vocational education supervisor at the Smithville Correctional 
Institution for fivfe years. The program was well established before Pat arrived and has been 
running essentially the ^same way for the last ten years/Although there are a few^problems, Pat is 
basically pleased w^ith the program. s * 

Motivated by the growing concern about accountability for public expenditures, the state* * 
education supervfsorjias mandated that all education supervisors of correctional institutions 
establish a procedure for evaluation. The supervisor explained that the information from the * 
evaluation' would be used to improve the vocational education program and to present to 
decision makers who fund the program. 
\~ 

I 

At first Pat was apprehensive about organizing and conducting evaluations of the Smithville 
vocational education program. Pat was concerned that the staff would feel threatened and the 
evaluation would be critical of the vocational education program. Pat also knew* however, that 
the program tiad strong points. Over the past year$, a nuiViber of former inmates had contacted 
the vocational education teachers to tell them about jobs they had found because of their 
training at Smithyffe. As Pat discussed the procedures with the state supervisor, they decided 
that ongoing evaluation would help document the strengths and provide information ta improve 
the Weaknesses of the progVam. 

Pat and the education supervisor decided to cpnduct a self-study first, and then to organize 
a program review team to evaluate 1 the program. The self-study would give the teachers a chance 
to identify "strengths and weaknesses and to make recommendations for improvements. Data 
would, therefore, be collected that pould be us,ed by the review team. 

Pat and the education supervisor at Smithville made plans to conduct evaluations of the 
program. They clarified the purposes of program evaluation in order to collect relevant 
information. Efforts would not be wasted collecting xinusable/extraneous information. The 
evaluation procedures would pertain specifically to the vocational education program rather than 
the prison industries or other education programs. Since Smithville is an adult facility, the \ 
vocational program wars designed to provide skill training for specialized jobs. The program 
included masonry, welding, carpentry, and auto mechanics. All o'f the programs were taught in 
the institution by vocational education teachers. 
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In order to collect both process and product types pf information, evaluation is approached 
in a variety of ways, such as through^program reviews and follow-up studies. Evaluations done 
«by internal staff are typically called self-studies or self-evaluation. Evaluations are also done 
through program review teams, or accreditation teams which typically include external as well as 
internal members. The seclkui, "Identify $taff" provides more informaUon,about the staffing for 
evaluations. The purpose of me evaluation will determine the way you approach the evaluation 
efforts/ 

j ' it 4 

Well-rounded evaluations include procedures to collect both product and process types of 
infocmation. Theemphasis will vary, of course, ^lepencjing upon the purposes for conducting the 
evaluations. In most cases, correctional educators must plan to eollect, analyze, and report both 
process and product evaluation. Typically, Both internal administrative decisians $nd decisions 
ma<Se in the state department and legislature will require that information be collected ^bout-the 
quality of the vpcationat program and the outcomes' of inmates' participation. i 

N / ^ 
- It is important to have clear purposes for evaluating the vocational education programs in 

your correctionaHnstitution. piear purposes will help you to avoid wastiftgjime, making costly 

mistakes, and incurm*^jj^iike for evaluation among the sta/f and administration. Moreover, 

clear purposes will help you write a useful report. Many evaluation reports are not used because 

? the results do not meet information'needs for planning and decisionmaking. 
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Use the following checklist to clarify the purposes of evaluating your vocational 
education programs (adapted from Wentling 1980). 



Deciding the Purposes for Evaluating 
Your Vocational Education Program 



Please,check yes or no. 

n . 

Do you need information for: * . 

/. Program Planning? 

Such information as the adequacy of the facility and the^equipment, the qualifications of the 
staff f 'the enrollments, and the numbers of inmates desiring to enroll, is useful in planning 
the types of programs that can be Offered, for expansion, and so forth. 

2. Decision Making? * ' 

Decisions, regardless of their size, must be based on a certam'amount of information. 
Decisions about assignment of staff to programs, and selection of inmates for participation, 
may ail require soVne justification. One way to make defensible decisions is to gather 
objective, evidence to support the decisions. ( 

3. Professional Development?! 

Evaluation can benefit, teachers, administrators, -and support staff. An evaluation system that 
identifies deficiencies and strengths in staff performance will help promote desirable changes. 

■* „ * 

4. Program Improvement? 

Improving vocational education programs is an important reason for evaluation. Evaluation 
activities should identify the actions, components, or characteristics that promote desired jor 
* unde^sired outcomes. Once program deficiencies have been identified through evaluation, 
action may be recommended to correct them. * 

5. Accountability? t t - 

Accountability requires a presentation of program results (such as placement of inmates in 
jobs, reduced disciplinary problems, and achievement scores) in relation to incurred costs 
and established desirable outcomes. I nternall^and externally these accountability measures 
can help corrections educators and admnistrators justify the resources spent for vocational 
education programs. 



& /(ccreditap'on? 



Evaluation of vocational education programs is often done to determine compliance with 
the accreditation requirements of the. Commission on Accreditation for Corrections or other 
accrediting agencies. 




YES NO 



□ □ 



. □ 



\ 



□ □ 



□ □ 



g □ 



□ 



D □ 



Keep the»purpose 'or purposes of your evaluation efforts in mind as you continue through the 
eight steps of evaluation. Most corrections educators have several purposes for evaluation and 
use evaluation results to convince decision makers to allocate more resources, to justify the 
previous aHooation of resourcesjo start new programs or expand existing ones, for accreditation, 
for public relations, and so forth*. ... 



18 



determine purposes 
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determine questions 

- i 

select methods 



identify staff 



collect data 



analyze data 



report results 




Smithville continued. 



The education supervisor u^ed Smithville's written statement of philosophy about the 
vocational education program to list the desired outcomes. _y 

i, 

Since the philosophy statement had not been revised for several years, Pat discus§ed the 
desired outcome^ with the education supervisor and with the warden to see if they were still in 
agreement. After reviewing the outcomes, Pat decided to rewrite them as more-specffic 
statements that could be evaluated. 
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identify outcomes 



The next>step in evaluatidn'is identifying the' desired outcomes of the vocational education 
program in yoiir correctional institution. There may be differences in desired outcomes among 
the various type§ of institutions. For example, juyenile and maximum security adult institutions 
will most likely have different desired outcomes for inmates who participate in the vocational 
education prpgram. In the National Study of Vocational Education in Corrections conducted in 
1977, by Abram and Schroed^r.the outcomes for vocational education programs were ranked 
from 1 (most important) to 7 (least Important). Table 1 shows the rankings of the stotpn 
outcomes by respondents from 93 juvenile and 260 adult facilities across the nation. 



TABLE 1 

RANKING OF OUTCOMES FOR VOCATIONAL 
EDUCATION PROGRAMS IN CORRECTIONAL INSTITUTIONS 





Ranking for 


i4 




Juvenile 


Adult 


Outcomes for Vocational Education Program 

9 


Facilities^ 


Facilities 


Develop specif ic job skills ' 


2 

« 


1 


Place inmate in a job on release 


4 


4 


Develop inmate's personal and social skills 


3 


3 


Develop inmate's work habits 


1 


2 


Provide a means of evaluating inmate for parole 


7 


6 


Provide inmate with constructive activity 


5 


S 


Other 


6 


• 

7 • 
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Desired outcome#can/be stated more specifically as objectives. At the J. F. Ingram State 
Technical Institute in Alabama, the specific objectives of-the vocational education program for 
first offenders are as follows: 

•1. To provide for eachjnmate enrolled a vocational educationrprogram applicable to the 
. individuaPs educational level, interest, aptitudes, and abilitils whicfi will enable the trainee 
tS meet the demands of potential'employers. 



2. To keep the trailing programs and facilities up to date and current with present day 

practices in thet>Qcypation by utilizing the advice and counsel of graft committees made up 

of persons presently employed successfully In the occupations. 

» 

3. To develop safe work habits arid practices by teaching and demonstrating safety in the 
performance of the work of the occupations. 

4. To make the training situations as nearly like actual employment as plfesible through 
providing the necessary equipment, tools, and materials needed and througK#uitable live 
work opportunities. ^ * 

5. To provide training which witl enable thejnmates to gain entJy level employment skills and 
knowledge in several aspects of an occupation. 4 

6. To provide counseling and other assistance designed to help the inmate to readjust to 
society and to th£ world oftoork. 

7. To instill into inmates a working knowledge of how to secure a job, retain that job and make 
^> progress in it. " - 

The -desired outcomes provide the framework for, and largely determine; the evaluation that is 
conducted. Therefore, the time and energy you spend in identifying outcomes and more specific 
objectives i§, very important. v 
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The next checklisf^f lists several possible ^ulpomeTfor vocational education programs in 
correctional institutions. Add your own desired outcomes to this list, then rank the outcomes. 
This ranking will help you determine evaluation qyestions. Rank the.yes outcomes from 1 (the 
mofUmportant) to the number of outcomes you have listed (the least important). 



r 



A 



Identifying Desired butcomes , 

for Your Vocational Education Program 



Please.check yes or no and rank the yes outcomes, ^ 

1 . Develop inmates' skills for a specific jo\>r type of jobs 

2. Place inmates in jobs related to the vocational education program ujjon release 

3. Develop inmates' Self esteem and self confidence 

4. Develop inmates' positive work attitudes and safe work habits 

5. - Develo primates' personal and social skills 

6. Provide a means of evaluating inmates for parole * \ 

7. Provide inmates with constructive activity 

& Reduce the number of former inmates returned to prison for lack of salable 
work skills in the ffee world 

9, Reduce the number of discipline problems among inmates 




□ 
□ 
□ 
□ 
□ 

□ 



10. Develop inmates' skills to look for a job, prepare a job application, write a 
-resume, and conduct interview 

11. Develop inmates' occupational goals and plans to attain the goals in tjy free world 

12. Provide inmates witMjrfihj^for entry level employment in jobs , □ 

13. Provide inmates with training that simulates real jobs, in the world with up-to-date 
facilities and equipment, necessary materials, realistic curricula, and qualified teachers □ 

14. Provide appropriate training for inmates' individual abilities^educationai level, 
interests, lengths of incarceration, and temperaments 



15. 
16. 
17. 

ia 



□ 
□ 
□ 
□ 
□ 



RANK 



□ 

□ 
□ 
□ 
□ 
□ 

□ 
'□ 

□ 
□ 
□ 

□ 

□ 
□ - 
□ 
□ 
□ 



Now that you have decided on the desired outcomes of your vocational education program 
and fflve ranked them in order of importance, review the top five. CheqkV these are really the 
most important outcomes desired for your program. These five (or more/less if you so choose) 
outcomes- will be the basis for determining the questions that will be asked in the evaluation. 
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determine purposes 



identify outcomes i* 



j - 

^Jetermin^^^ 



select methods 



identify staff 



collect data 



analyze data 



report results 
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Smithviiie continued . . . ^ t 

Once the desirable ou#omes had been agreed upon, l*at had to develop the questions that 
woufd indicate whether Qr not the out^jgtes had been achieved. The questions had to be measur- 
able or specify the data needed. 



Pat 's Questions for the Evaluation of Desired Outcomes 



. Desirable Outcomes 

1. Develop inmates' skill for a specific job or 
type of jobs . ? \ 



Z Develop 9 inmates' desirable work attitudes 
and safe work habits 



3. Place inmates in jobs related to their 
vocational education progrwnupon 
release 



4. Provide inmates with training tyat 
simulates reaf jobs through adequate 
equipment and facilfties/tfecessary 
materials, realistic curricula, and 
qualified teachers 



5. Reduce the number of former inmates 
returned to the correctional institution 
for lack of salable work skills 



Possible Questions for Evaluation 

What skills have inmates attained as a result of 
their vocational education training? 

How much economic advantage have inmates 
gained as a result of their training? 

Do inmates enrolled in the vocational education 
p/ograms demonstrate ^desire to work? : 

Do inmates perform job tasks correctly and use 
tools and equipment safely? 

Are former inmates working in jobs that require 
and use skills they learned in their vocational 

nn nmnram? 



education program? 



Did the correctional institution help the inmates 
acquire a job related to their vocational educa- 
tion training? 

"Are the equipment and facilities adequate for 
learning job skills necessary in the free world? 

Are adequate and appropriate materials available 
for learning realistic job skills? 

is the curriculum current and relevant with 
industry and business practices in the regiort^ 
where the inmates will be released? 

Are the vocational education teachers certified 
as such with necessary qualifications to teach 
inmates for the free world? 

What proportion of fSrmer inmates who had 
vocational education training returned to a + 
correctional institution within a year compared 
to those who did not have the training? 

How many curretft inmates have had no voca- 
tional education training priof to incarceration? 
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The third step in evaluation is determining questions based upon the desired outcomes. The 
traditional evaluation question jsrTo what extent Is the program attaining the desired outcome? 
n the following worksheet", list the outcomes you identified as most important for your \ 
stituti^n's program. Think about the questions implied by the outcomes. Write at least "one 
question for each outcome in the space provided. 

~ Worksheet 1: Determining Questions 



Desirable Ovfcorqfs for Ybur 

Vocational Education Program Possible Questions for Evaluation 



1. 


■f 

* 




«% 


3. 


\ 


4. 


4 

4 


• * 

5. ' 


— ^ : 

• 



The questions you have listed above will determine the types of methods you will select to 
collect data for ydur Evaluation. 
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Smithville continued. 



■ The first time the evaluation committee met, they decided what information was needed to 
meet the objectives of the evaluation and how to collect it. As they talked the* realized they ha<4 
tome information already, but it was scattered and not orgamzed in a usable manner. Pat had 
enrollment figures for each program area, the number of leavers and completers, and the number 
of inmates waiting to enroll. The recprds also indicated whether those who left without 
completing were paroled, were disciplined, or had chosen not to finish. If Pat or the education 
supervisor had not had these records, the committee w6uld have recommended that they start 
being kept for future use. 

teachers also had scattered pieces of infaufcdtion that could be used, such as records of 
employers who requested references for fapfner inmates and records of inmates who had 
maintained contact with them after release. Collecting existing information required a lot of 
detective work, but it produced lists of employers and former inmates useful for needs 
assessments and follow-up surveys «, 

One of the evaluation questions was whether forme^inmates who had completed a 4 
vocational program were-employed in occupations related to their training. The evaluation 
committee realized however, that follow-up surveys of former inmates are difficult.to conduct for 
severaF reasons. 

» \ 

• Former inmates typically return to their hometowns; this covers a wide geographic region. 

• Records of their addresses are often incomplete, incorrect, or nonexistent. * 

•> Communications between parole offices and corrections institutions are often not strong or 
dependable. ^ 

• Former. irtmates want to forget their imprisonment ancj do not usually want to participate in 
a follow-up from the correctional institution. w 

A member of the evaluation committee suggested that they contact p^ole officers to see if 
some type of working relationship could be established to provide systematic follow-up 
information. Pat realized that it would be very difficult to obtain correct addresses for a follow-up 
and made a note to investigate ways to coordinate records for future use dlth the parole boar^l 

To answer some of their questions about the quality of the program, the committee needed 
to know what was actually happening while inmates were, enrolled in the program. They decided 
to suYvey inmates and teachers to determine their perceptions of the facilities, teaching methods, 
curriculum, suppo/t services, and general impressions of the usefulness of the program, The ' 
program review team's observations and the information from the inmates would be compared to 
' identify problems and strengths. 

Pat wrote an inforrtT&tion sheet describing the operational policies and procedures regarding 
the selection of inmates, criteria for admission, services offered, course requirements and ~ 
contents, and so forth. The information sheet would also include enrollment data, number of 
teachers, the number and tyjJfe of major equipment, and the number of students using the 
support services, .such as counseling. The information sheet would give teachers and 
administrators an overview of the pr&gram,*would be important in orienting the program review 
team, and would provide background information for the evaluation reports. 
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Step 4 

select methods 



The next step in evaluation is selecting the methods to answer {he questions. There are two ; 
basic types of methods. The first type'measures in rtumbers and statistics and includes such 
information as enrollment and attendance figures, placement data, and recidivism rates. This 
\type of method is called quantitative. Th^data is collected by tabulating and analyzing records, 
questionnaires, tests, checklists, and so forth. 

The second type does not measure in numbers, but includes such information as description 
of facilities and usefulness of equipment, interactions between inmates and teachers, content of 
the curriculum, attitudes, opinions, 4 and behavior patterns. This type of method is called 
qualitative and is collected through observations, interviews, and reviews of records. 

0 

The major methods for collecting quantitative information for evaluation are as follows. 

* Records— Existing records or documents can provide much usable information but are often 
scattered among several office^TPnrollment figures, numbers penmates completing s 
program, number who left the program, and information on r^pfeat offenders, grades, test 
* scores and interview, sh^d demographic information shoCild be available from the tiles-of 
administrators and teacher 

1 Questionnaires are a set of printed questions that can be mailed or distributed in a group 
such as a class. Questionnaires can be answered anonymously. Because of their structure, 
they arej&psy to tabulate by hand or by computer. Although they are generally easy to 
administer to inmates and staff, mailed questionnaires can present many problems in 
corrections. Names and addresses of former inmates and employers are frequently not 
available. EVen when available, most former inmates do not return questionnaires to a 
correctional institution. Answering questionnaires is often a problem for former inmates who 
have relatively lo^^ducational levels and limited reading ability. 

Tests can be standardized or developed by staff. There are many kinds of standardized tests 
available commercially to measure the following: ) 1 

Intelligence . I 

Personality 

Aptitudes . \ 

Interests ' 
Achievement 
Knowledge?* 
Social adjustment* 
Goals 
Skill leveT 
Job performance * 

Attitodes * * 
Work values 
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Tesj£ are used to.assess an inmate's potential and progress in vocational education. When 
affecting standardized tests, it is important to consider the population for which they were 
designed. Again, reading ability is a factor that may interfere with using tfests with some 
inmates. Some tests require hands-on performance to assess the innfate's ability to $6 a job 
, or perform a skill. 

Checklists are often used by evaluators to cover a large number of items qu#f<fy Checklists 
are useful in self-studies of the institution or for monitoring regulady scheduled activities. In 
this handbook, checklists are used to present lists of ideas and reniknJers. 

The major methods for collecting qualitative information for evaluatiog are as follows: 

j Observations— The direct observation of persons in a group program has some obvious 
advantages, such as the opportunity to view the. group in its normal environment and the 
opportunity to observe firsthand the emotions and attitudes of program participants. One 
problem is that when* observations are made without the observer knowing the situation 
previously or the situation to come, there is.no way to know if they represent a true picture of 
what is really happening. There is also a possible problem that the presence of the observer may 
Cause the persons-being observed to change their typical behavibr. A series of observations over 
.a period of time can lessen these problems somewhat. 

Interviews— Interviews are effective with persons who have reading difficulties or when inTdepth 
infQrmation is needed. However, interviews can be costly and time consuming to do. It is 
sometimes difficult to summarize the information from a series of interviews. They can, however, 
- be effective when used in combination with other types of data collection and when seeking the 
answer to difficult, complex questions. Interviews also provide the opportunity to see how the 
person reacts to questions. The "body language" of the persofi often tells the interviewer more 
than the spoken answers. 

Records— Existing records or other types of documents can provide descriptions of programs, 
events, activities, and so forth. Newspaper articles, teachers' anecdotal records, previous 
evaluations, reports, proceedings of meetings, lesson plans and curricula are some examples of 
records useful for qualitative evaluation. , 

All o^ these methods of collecting information are useful to answer evaluation questions A 
combination of quantitative and qualitative methods are used in many evaluations. For example, 
using a combination of methods is most appropriate to conduct a follow-pp of former inmates 
t enrolled in the vocational education programs A mailed questionnaire (Sample 4), telephone 
interviews with nqnrappondents, and an examination of parole records might be used to follow > 
Up former inmates. (A detailed discussion of the follow-up is discussed in the "Collect Data" / 
section of this handbook.) 



it 
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* Use .checklist ^» to select the methods most appropriate to answer the evaluation 
questions you wrote in the previous section. Remember that some questions may be best 
answered using a combination of methods. , 
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Selecting Methods to Conduct the Evaluations 



Please write your questions and check the methods most appropriate for each question. 

Your Questions for Evaluation Most Appropriate Methods 




1. 



□ 1 Review of records □ Observations 

□ Questionnaire(s) □ Interviews 

□ ' Test(s) □ Checklists 

□ - Otfter: 



□ Review of records □ Observations 

□ Questionnaire(s) □ Interviews 
JO; Test(s) ■ □ Checklists 
'□ Other: * 



□ Review of records □ Observations 

□ Questionnaire^ □ Interviews 
£□« Test(s) □ Checklists 

□ Other: 



□ Review of records □ Observations 
Questionnaire(s) D Interviews 

□ , Test(s) □ Checklists 

□ . Other: . 



□ Review of records □ Observations 

□ Questiqnnaire(s) □ Interviews 
□ . * Test'(s) ' □ Checklists 
'□ Other: ' 



Keep the methods you selected to answer the questions in mind as you proceed through the 
handbook. The methods you have selected will be important when you identify staff in thejiext 
section. ^ 
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collect data 



analyze data 



report results 
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Smithville continued, . . 



Pat and the education supervisor had already decided to conduct a self-study first, and to 
later organize aerogram review team to evaluate the program. 

They held £ meeting with all oMhe vocational education program teachers to present the 
evaluation plans. After much discussion, the teachers made suggestions to carry out the plans, 
and a timetable was established. Two marfibers of the $taff agreed to work with Pat as an 
evaluation committee to develop the self-study procedures and the questions and to organize 
the evaluation efforts. 

It was important for the program review team to be familiar with corrections and each of the 
vocational areas being evaluated. A former inmate was included on the review team because he 
could offer insights based on.his participation in the vocational program. The program review 
team would be composed of a current inmate, a former inmate, a local, employer yvho had hired 
former inmates, an instructor, and the state vocational education silpervisor for corrections. 
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- An important step in evaluation is identifying evaluation staff. The people who conduct 
evaluation are important because they influence the cost, quality, credibility, and usefulness of 
the results. 

Frequently, the cost df evaluatipn is not budgeted for vocational education programs in 
correctional institutions. An accepted estimate of the cost of evaluation is approximately 10 
percent of the total budget for the program. This may not be possible in the majority of 
correctional institutions where the budget for Vocational education programs is very low or 
nonexistent. 

There are various ways to staff the evaluation of vocational education programs in 
correctional institutions. ♦ N 

\ 

Internal Sfaff— In many cases the education supervisor and vocational education supervisor 
have the re^>onsib|lity.to provide Evaluative information to the institution's administrators 
and to state administrators. A staff member or a committee is appointed to coordinate the 
evaluation activitiesLThe self-evaluation is often done before an evaluation is done by a 
review team. A self-evaluation usually involves little cost but the objectivity may be 
questioned. 

Professional Consultant— Hiring a professional consultant from outside the system solves 
the problem of objectivity. However, consultant services can be expensive, and finding a 
consultant familiar with vocational education programs in corrections may be difficult. 
* 

Program Review Team— A program review team consists of people with expertise in a 
technical field, vocational education, corrections, and so forth. The composition of the 
program review team js determined by the purposes of the evaluation. If tKS team is being 
used to validate information from a self-study, then knowledgeable people from the 
community, Such as employers, former inmates, and community leaders, should compose 
the team. But if the team is doing the entire evaluation, then persons from both inside and 
outside the system need to be selected. The team should include a broad representation of 
expertise. The following priteria are useful for selecting team members: 

• Experience in evaluation 

• Experience in vocational education 

• Expertise in teaching, administration, or supervision 

— — \ • Awareness of the philosophy of vocational education in corrections 
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• Understanding of the uniqueness of vocational education In correctional education 

• Understanding of employers 1 needs 

• Recommendations by colleagues 



The major pitfall that hampers the credibility of persons selected to evaluate is role conflict. 
A person who has helped develop and operate a program is understandably biased and would 
want to present it in the best light. In some cases the evaluator's job security or status could be 
threatened by unfavorable evaluation results. Credibility is important and can be increased by 
hiring a professional consultant or using teams of persons not directly connected to the* 
institution. Usefulness of evaluation results, on the other hand, is usually increased by involving 
staff in the evaluation effort. The best option is assigning a Staff member the major responsibility 
for organizing the evaluation, hiring a professional consultant to help design the evaluation, and 
assembling a team for a thorough review every three or so years. 

t 

There are important qualifications to keep in mind when assigning staff, hiring a professional 
consultant, or organizing a program review team. 

5> 

The next checklist"^/ lists the important qualification^} order to help you decide about \ 
the people you are considering to staff the evaluation activiros. 
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Determining Qualifications for Evaluation Staff 

Please check yes, no, or. don't know. * * ~~w^m, m vj 

Does the person you are considering as a consultant or as a member of an evaluation team have- 

s. 

/ 1. Knowledge about evaluation^research problefns, literature, and strategies? 

2. Knowledge about evaluation methods that have been used in* . ' 
corcactional institution? * 4 V . . * ^ * ' 

3. Ability to develop evaluation questions that can be measured? 
, 4, Ability tt> organize and corfduct evaluations in a correctional institution? 

5. .Ability to report evaluation results and to make Recommendations in ^ 
a tiWly manner? v \ Jflfc, 

6. Knowledge of measurement with the ability t^lbcate and select tests? 

7. Ability to develop testJ^ftitude scores, interview schedules, and surveys 
if necessary?. « 

8. Knowledge of statistics to tabulate arid analyze data'from evaluations? 

9. Knowledge of data processing? - M 

10. Knowledge about the operation, political factors, and constraints of the 
vocational education programs? * 

1J. Knowledge about vocational education curriculum aiiti possible outcomes? 
*8.* Freedom from vested interests in the results of the evaluation? 

13. Reputation-and status in the professional community? *^ 

14. Trust and respect of the vocational education program staff? 



YES. 


NO 


DONT 
KNOW 


□ 


□ 


□ 




□ 


□ 


□ 


□ 


□ 






□ 


□ 


□ 


□ 

~ r 


□ 

V 


□ 


□ 


□ 




□ 


□ 




□ 


□ 


□ 


□ 




□ 


'□ 


□ _ 


□ 


□ 


□ 


□• 


O 


o 


Q ' 


D* 


P. 


□ '" 



$ince no one person or group of persons are likely to have'alF of the quafifications, select 
persons who have-the*most qualifications. Compose learns of persons with $ cross-section, of 
qualifications.' h • ' 
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The following checklist ^/.provides suggestions for the types of persons you^pjght select 
for a program review teanj.Jdentify people in the community and institution that bestjmeet the ' 
qualifications in checklist 5. ■ 




Identifying Program Revfew Team Members 

% 

Check the types of persons needed on your team; then list persons you are considering for the team. 

i 

1. Community Representatives 

□ Local community leader * ^ 



Name/Telephone Number 



□ Local employer 

□ LocaLunion leader 



□ Local minister/priest/rabbi 



□ Employed former inmate 

f <□ r County agricultural extension agent 

% * - y 

□ Others * 

2. Institution Representatives 
< n Education director 



d Vocational supervisor 



d Vocational teacher 



2 Inmate vocational student 



ERLC 



It will be necessary to allow time at the beginning of the evaluation effort^p orient the 
evaluation staff. Professional consultants need to becortie famijiar with the institution. Those 
without experience in corrections neeW to become aware of the Special problems involved in 
providing vocational education in a correctional instittjJiDn. All persons involved in ev$ uation 
need to understand the purpose for doing the evaluatid^and how the results will be us^d. 

• * . ' . 40 



determine purposes 



identify outcomes 



determine questions 



select methods 



i 



identify staff 



collect data 



analyze data 



report results 
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Smithvllle continued. . . 



The committee identified, several pers^jyhat feould provide information about the vocationar 
education program and listed the types of WTCrmation each one might offer. The education 
supervisor hid the results- of checklist type of survey. It had been developed for statewide use 
to determine it correctional institutions were meeting the minimum requirement for vocational " 
education pro'gram facilities an# equipment. Since the survey had been completed last morjth, 
the supervisor suggested using the results in \he evaluation. The committee selected- several 
instruments, including a survey fornrTfor inmates, one for teachers, and a checklist for 
accreditation, and reviewed them for usefulness. 

During a second meeting with the ^^fe^the evaluation comnruttee presented the plans for- 
evaluation of the program/taacheris were fiSfl^flo tontribute any-notes or records they might 
have kept and to participate^ the faculty survey- The teachers agreed to conduct the inmate 
survey during class time arid discussed Jhtf problems of the inmates' reading abilities that could 
affect the survey. A teaches suggestion that inmates be given the option of completing either a 
written or oral survey was 'accepted. Th#^mmittee decided to ask the chaplain who was trusted 
by inmates t9 ^nduci anyf pfal surveys glreH to inmates. *. 

While the committee w$s f}»h^rinjg evaluation information; Pat contacted potential members 
of the program review teamfe seef ii they wouldBagree to serve. A date .for the team review was 
set for three .gponths in advance ta allow Mnpj^sr the staff to conduct the self-study. 

Since all of the review?teanr\ members lived within tv£ hours of Smithville, the first meeting 
was set for ID;00 a.m. The/Purposes of t«e evaluation, thrquestions the evaluation would answer, 
a backgroOnd information sheet, and a summary of data oollected to date were mailed tp them in . 
advance" of the program reftritipE days. 

i 
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" AGENDA 

Review Team Evaluation of the Vocational 
Education Programs at Smithville Cprrectional Institution 



Tuesday 

9:30 a.m. 

10:00 a.nru 
*10:30a.m. 



COFFEE 



Welcome and Introduction* 
Lee Dims, Warden, Smithville 
Tracy Mann, Smithville \ 
Education Supervisor 

Orientation 

Pat Howell, Smithville 4 Vocational 
Education Supervisor 



11:30a\m. LUNCH 



1 :00 p.m. Observations of vocational programs 
Team A — Welding 
Team B — Auto Mechanics 
k Team C — Agriculture 

6:00 p.m. DINNER 



Wednesday 



7:30 a.m. 



8:30 a.m. 



9:30 a.m. 



Noon 



BREAKFAST 



Discussion of Jesuits and planning 
for Day 2 



Group and individual interviews 
with teachers, inmates, administrators, 
and related 1 

Compilation of results and LUNCH 



J 



4:00 p.m. Preliminary presentation of findings 



A chance for the review team to get acquainted with 
each other and to meet the Smithville staff. 



Explain layout and operational procedures^ Smith- 
ville. Describe vocational p^grams offered, reasons 
for evaluation, outcomes of program, and how the 
next two days will be structured. 

During the group lunch furnished by Smithville, team 
discusses plans and asks questions. ^ 

Teams observe classes in session, walk through all the 
facilities, and interview teachers and students. 



Warden, education supervisor, Pat, and the evaluation 
committee are included. This is a chance to answer 
informally questions that came up during the day. 



Pat and the evaluation committee are present to answer 
questions, arrange fpr additional observations or inter 
views if needed, and to provide information requested 
by^team. 

Only the review team is involved. Members review the 
data thus far and identify what information is still 
needed. 

Pat arranged these as requested by the review team. 



Lunch is onjyfw review team so they can continue 
working. 

The group meets with Pat, the eva^luation committee, 
the warden, and education supervisor to present pre- 
liminary findings. They will complete their reports and 
return them in written form within two weeks of the 
visit. 
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Once the staff is selected, the data collection can begin. The folfowing guidelines are 
important in collecting data: 

• Use methods and instruments that are suited to the evaluation questions. Collect only the 
information that is needed to answer the questions. Collecting extra information is costly, is 
a bother to respondents, and gan detract from the real purposes of evaluation. 

• Use more than one method of collecting information when assessing important issues. The 
methods used in evaluation can always be challenged. Therefore, use more than one 
method or instrument to collect the same information. 

• Use professionally developed tests to assess sensitive personal issues, such as- vocational ■ 
interestST^alues, attitudes, intelligence, or psychological states. Professionally developed 
tests are available from commercial publishers. Several thousand of these test? are 
described in Buros' Eighth Mental Measurements Yearbook, which is listed in the 
re f ere nces. Over 200 tests are described in Career Education Measures: A Compendium of 
Evaluation InstrGtnents which is abstracted in the resource section. 

• Keep information collection as unobtrusive as possible. The evaluation efforts should, not 

infringe upon the normal operation of the institution or take too much of anyone's time. 
p 

A number of persons or sources can provide the information through various evaluation . 
methods. It is important to know who can provide the information to ansy^r the evaluation ^ 
questions. Generally, several sources are needed to give a complete picture of the question 
being asked.^For example, teachers and inmates may have very different view points of the 
ifsefulness of the counseling senlbes provided or the adequacy of the equipment available for 
the classes. Collecting data from several sources reduces the possibility of distorted results. % 
When there are several conflicts between two sources of information, additional 90urces should 
be investigated. 
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Use the next checklist "A/ to help decide who can provide information for evaluation. 
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Identifying Sources of Evaluation Information w 

Please check yes, no / or maybe. 
Will you collect information from- 

/, Present Inmates 

• Evaluation'fcf-teachers 9 * 

• Satisfaction with teaching method 

• Student test scores {achievement, aptitude, attitude, competencies, etc.) 

• Curricula and instructional methods used 

• Adequate facilities, materials, and equipment 

• Satisfaction with support services (counseling, library, etc.) 

2 Former Inmates 

• Working or unemployed , 1 

• Job related to training 

• Usefulness of vocational education training 

<3. Vocational education teachers 

• Performance and achievement of students 

• TeacJter qualifications 

• Curricula and instructional methods used 

• Adequate facilities, materials, and equipment ^ 

. Vocational guidance counselors 

• Number of inmates using counseling services v. 

• Description of vocational counsel in^offered 

• Placement statistics 

• Number of employers contacted 




□ □ □ 



□ 



ing^off* 



5. Vocational education supervisors 

• Teacher /pupil ratio % 

• Teacher qualifications 

• Curricula and instructional methods used 

• Cost-effifciency of vocational education program 
4 • Adequacy of facilities, materials, and equipment 

6 Employers of former inmate$ 

• Number of contacts with placement service 

• Satisfaction with inmates' training 

• Satisfaction with skills taught 

• Assessment of skill needs 

7 Vocational advisory committee 

• Current ness of curriculum . 

• Adequacy of facilities, material, and equipment 

• Relevancy of skills taught 

8. Others (specify:) ■ 



□ 



a □ □ 



□ □ □ 



□ □ □ 



□ □ □ 



□ □ □ 
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Observations 

All .methods of collecting data are simply different forms of observation. Observation as a 
method refers to watching and recording what i$ said or done or looking at th^ facilities and 
equipment against a set of evaluation standards. Your presence in a class can influence what is 
happening. S0fl|e points to remember when observing: 

• ♦ Try not to be self-conscious and caJI attention to yourself by trying to "hide." 

• Take brief notes rather than wril'mg a lot while in the class. ' 

• Participate if the situation calls for it rather than calling attention to yourself. * 

• Avoid misinterpreting an incident, such as anargument, as being the normal situation. 

• Try to o^erve situations more thai>once to check your perceptions. 

Checklists are often used to record observations in self-study evaluations and when using 
program review teams. Ike evaluative criteria for facilities, equipment" materials, and classroom 
maftagenftent are usually in the form of checklists. Checklists are provided by either the state 
department of corrections, vocational education, or education. The example section of this 
handbook contains a checklist form adapted from the Minnesota Department of Corrections. 
*■ * 

Observations can help in evaluating the quality, quantity, and amount of use of the facilities 
and equipment. Inmates spending large amounts of time waiting to use equipment may indicate 
a problem with the facilities arrangement or class organization. Equipment with missing or 
broken parts may indicate a &rious problem with facilities upkeep, or it may be a temporary 
condition that is in the process of being corrected. 

Interviews 1 

Interviewing, or talking to people, is one of the most ccynmon methods of collecting 
information. Inteviewing rs more than simply asking people/questions." 

Collecting complete, accurate answers through interviews does not just happen. A certain 
amount of practice is needed. It is important to make respondents feel relaxed and comfortable 
during the iriterview. They should not feel threatened in any way, either by the presence of the 
interviewer or the way the interviewer asks the questions. 

The following are some suggestions compiled from Illinois and Colorado program review - 
handbooks for conducting interviews: 

• Be a sympathetic, attentive, and interested listener. Encouraging gestures, such as nodding 
and smiling, and words, such as "I know how you feel," help show the respondents you 

* appreciate their remarks. 

• Be neutral with respect to the comments and subject matter. Don't express you£ personal 
opinions on the topics being discussed. This could discourage the respondents from 
speaking freely. Your job as an interviewer is to understand and accept what is being said, 
rather than to approve or disapprove it 
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Be friendly and informal but alw^sprofess/ona/. Everything that you say or do should. help 
overcome the respondent's fear of sharing feelings. 



• Be observant. Watch for the gestures that^respondents use to express thecp^elves. Thesfc, . 
serve as cues that ttoe respondent* are Uncomfortable dt ill at £ase during the interview. A 
change of topic /nay assist the respondents in overcoming the discomfort. Ask sensitive 
questions after you have Established rapport. 

* 

• Be at ease yourself. Do not hurry the interview or appear hesitant, embarrassed, or 
awkward. The respondents would soon sense your discomfort and behave accordingly. 

\ 

There are two basic typos of interviews. In the first type no prepared questions are asked. 
The questions are based on the topic but flow according to the respondent's answers. The 
purpose of this type of interview is to obtain highly personal^and in-oepth answers. 

In the- other type of interview a questionnaire is used. The interviewer asks the questions in 
person or over the phone. Although this is somewhat limiting, there are opportunities to ask for 
more details when necessary. A great deal of information can be asked in a short time with this 
type of interview. Another advantage is that the respondent is not required to know how to read 
a questionnaire. ,\ 



Records and Documents * 

Useful information is typically available from records kept by various administrators and 
teachers. In most institutions there are a rftmber of documents that contain statistical data for 
comparison through the years. For example, enrollment data from the past ten years can be used * 
to find out what courses were not popular with inmates. Interviews with inmates can lead to 
r answers as to why a course has not been selected. The evaluation report can recommend 
changes to increase enrollment or to drop the courses entirely. 





In additio/i to institutional and state reports, other useful records include the following- 


• 


Minutes of meetings 




• 


Correspondence files 




• 


Demographic data 




• 


Inmates' student files 


<-* 


• 


Lesson plan books 




• 


Grade books 


■ ^^^^^^ 


• 


Curriculum guides 


• 


• 


Annual plans 4 




• 


• 

Facility layout diagrams 




• 


Organizational Charts 


C" 
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• .Former evaluation results * 

• Teachers' performance ratings - -* 

• Absentee records : 

• Budget records* 

Before using records, be sure of the questions you are seeking to "answer. It is very easy to be 
carried away when reading records and to gather useless information. 

In addition, anecdotal records, or notes about events as they happen, provide a wealth of 
Information that is not fbund in official reports. Teachers and administrators should be 
encouraged to keep their own anecdoctal records on a continuous basis. The J. F. Ingram State 
Technical Institute for Inmates in Alabama offers ajSuggestiOn in figure 2. 



FIGURE 2 

EXAMPLE: HOW TO KEEP ANECDOTAL RECORDS 



One way a teacher can keep up with students ts-ttvough use of a 3" x 5" card system A 
small box file would be adequate to hold a 3" x 5" bard on all students for/three years 

As a studentAfvters your class prepare a card with just the student's name and date 0 y 
enteredpirtffOne section of the fHe would contain all students presently enrolled Cards 
i#rflew students would be filed in alphabetical order in this section 

* 

Another section of the file system w>uld be for wtftiarawaJs When a student withdraws 
and is not issued a certificate or a diploma, the card is pulled from the^ section present • 
enrollments and placed alphabetically in the withdrawal section with a notation of why the 
student withdrew, where the student went, the date, and any other information available 

Another section of the file would be for graduates As a student completes the course 
pull his card from present enrollments section, enter on it the date of graduation, whether 
released or not and whether employed in a related occupation, or in an unrelated 
occupation, or is unemployed. Then as you find information about the student during the 
foljpwing two years it can be recorded on the card. 
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Mailed Questionnaires 

Mailed questionnaires are most often used for student and employer follow-up and needs 
assessments. A high response rate is needed for the results of mailed questionnaires to be 
meaningful. If the evaluation is an ongoing process in which questionnaires are mailed to a 
^jfoup of former inmates each year, then the current inmates can be alerted to the fact that they 
will be receiving a questionnaire in the future, and the purposes of it can be explained before 
they are released. A mailing address should be obtained prior to inmates' release. Inmates often 
fear that answering the questionnaire wiiMn some way affect their employment or cause other 
repercussions. Ever/ effort should be made to alleviate these fears by explaining that the 
information will be completely confidential and used to improve the vocational education . . 
program. . % 

Since most former inmates do not want anything to do with the correctional institution once 
they have been released, mailed questionnaires should not carry the retjjrn address of the 
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institution. Instead, arrangements could be made to use an advisory committee member's 
business address. Jf this is not possible^ Rost off ice box should be rented, preferably where the 
postmark wijl not be connected with the institution's address. 

\ The number of mailings will depend somewhat OA the money and time available. The 
suggested mailing below would be ideal if the institution's budget allows it. j 

1. First week: send a post card that alerts the former inmate that a questionnaire is coming. 

2. Second week: send the questionnaire, cover letter, stamped self-addressed return envelope. 

■ 3. Third week: send a thank-you reminder card. 

v4. Fourth week; send a second request for questionnaire, second cover-letter, and stamped 
self-addressed return envelope. 4 9 

5. Fifth week: send a second thaftk-you reminder caFd. 

6. Sixth and seventh wee^s: send third request and reminder cards. 

7. Send a thank-you letter once the questionnaire is received. 

, Studies show that repeated maNjngsdo improve response rates, but the costs must be 
weighed against the'advantagespf"increa§ed responses. At least two mailings are recommended 
and can be followed with telephone calls to these who do not respond. 

^ Checklist* 

Checklists are a > relatively quick method of evaluating many facets of a vocational education 
program. They are often used by accreditation, self-study, and program review teams. The 
example section. contains a checklist for accreditation standards. This checklist was adapted for 
users of this handbook from the sets of standards developed by the National Center for Research 
.in Vocational Education and the American Correctional Association. The example section also 
contains a'checklist useful for a self-study or program review. It was adapted from a checklist 
used by the Minnesota Department of Corrections. 

t 

Follow-Up Studies 

Each evaluation question may require several methods as discussed in the selecting methods 
section of this handbook. To answer the question, "Are former inmates working in jobs that 
require and use skills they learned in their vocational education program?" a follow-up may be 
conducted of former inmates. " 

Former inmate and employer follow-ups can provide data about the results of vocational 
education training (see samples 4 and 5 in Sample Materials section). Some key questions'trat 
can be answered by follow-ups include the number of former inmates who sought employments 
and are working, the relationship between a job and vocational training, employer satisfaction ' 
with former inmates, and^the attitude of former inmates toward the vocational education 
program. Follow-up studies of vocational education students in public schools are recommended 
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at one, three, ancHive year intervals to assess the long term impacts of vocational education 
training. This schedule is also recommended 1or correctional education if possible, although the 
constraints and cost§ may be prohibitive. 

The most frequently used method for follow-up is the mailed questionnaire. Follow-ups of 
former inmates are often difficult smce inmate records frequently lack*necessary information or 
are nonexistent. Because correctional institutions typically serve a large geographic area, such 
as an entire state, mostjormer inmates leave the immediate area, which makes it more difficult to 
find and contact theffiTFurthermore, as many corrections educators have realized, even if forijier 
inmates can be located, they may^not respond to a mailed questfonnaire. 

If it is possible to obtain names and addresses of former inmates,^ntfcntives should be 
provided for returning the questionnaire. Cash or other awards, such as restaurant certificates, 
have been successfully used with former inmate followwjp studies. 

The most effective method, according to corrections educators, is to develop a working 
relationship with the parole board and key parole officers across the state. This kind of network 
is most likely to provide current information about the location and activities of former inmates. 
Through parole records and by administering questionnaires through parole officers, more 
evaluation information can be collected. * 
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Uee^he following checklist"^/ to increase the response rate for your follbw-up study. 
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Increasing the Response Rate for a Follow-Up Study 



Please check yes or no. , S • yes no 



To ingress* the response ratel did you- 

1 . Inform inmates before they are released that you will send them a folio w-up . □ □ 
survey? .Tell them why you need the information and what you will ask. : 

Stress the Importance of their cooperation. v 

2. Promise complete confidentiality? Print^ny necessary code number on □ □ 
the back of the envelopes, very discreetly. v 

3. Provide a prepaid return envelope? '> . □ □ 

4. Include incentives if possible? Money is the best incentive, but small gift □ □ 
Alms can be used too. 

5. Use short surveys, asking only necessary questions? m O JZI^ 

6. Avoid asking personal questions that can identify the* respondents? □ D 

7. Place difficult questions last? O GD 

8. Use colored paper with very easy to read print? * □ JIp 

9. Use a personalized cover letter signed by a former teacher, trusted □ , □ 
counselors, or chaplain? 

10. Write "please forward" jon all mailings? , □ □ 



Another type of follow-up is the personal, face-to-face interview. The interview provides the 
chance to get a more detailed viewpoint on a former inmate's ideas. However, personal 
interviews can be expensive! and time consuming. A number of different follow-up methods have 
been tried, such as hiring fcjrmer inmates to find and interview others. At another site consultant 
evaluators invited former inmates to jom them for lunqh at a popular restaurant while they were 
interviewed. I # 

According to Nickens, Purga, and Noriega, telephone interviews are effective with most 
populations and more cost efficient than mailed questionnaires or face-to-face interviews. If it is 
possible to obtain current telephone numbers, telephone interviews are usually less threatening 
to former inmates. They arte more willing to answer questions over .the telephone than during a 
face-to-face interview. The amount of information that can be obtained over the telephone is 
more limited, however, than in a questionnaire or face-to-face interview, interviews are 
sometimes done in combination with questionnaires, either to provide mpre detailed information 
or as a method of reaching nonrespondents. 
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determine purposes 



identify outcomes 



determine questions 



X 



select methods 



• ? 



identify staff 



\ 



collect data 



report results 
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Smithville continued. 



The comfnittee organized all of tjie data from the self-stucfy into a notebook for use by the_ 
review team. The notebook included an organizational chart of the administration 
vocational education program and charts of the total enrollment in vocational education 
prog/ams for the test ten yfears and th£ current enrollment by program areas. 
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analyze data 



The data collected through various evaluation methods must be tabulated and analyzed so 
that theycan be understood and presented in a useful way. Detailed techniques to analyze data 
are available io the references listed in this handbook. Much of tne data analysis is fairly easy, 
however, and can begone by persons without experience in statistics. 

■' ^ -Hand tabulation and data processing by computer are two methods of iabulating data for 
; analysis. If you plaa to use cdmputer pro^ssing^hav^a programmer chfeck the questionnai^s 

9 or other forpis before you use them. The fomiats^Tihe dala collecti9n. instruments are very * 
important in computer processing. If the instruments are designed properly, they can easily be 
used for computer analysis. Some inmaje records may already be available on computer records 
and can be helpful in tabulating enrollments, for Example. 

. - 7 ' ■ . ^ 

Most data from evaluations bf vocational education programs in correctional institutions can 
be hand tabulated. For example, hand tabulation is used to simply tally the responses from a 
4 questionnaire. If you do not have too many questionnaires, you can easily record the number of 

responses on a copy of the questionnaire. Wh§n using checklists, count the number of yes and 4 
no resppnses and nonresponses to each item. It is important to report the- number of * 
nonresponses, which can indicate that respondents did not understand th£ item or chose not to 
' respond because it dealt with a sensitive issue. *l 

i ^Analysis consists of Various types of comparisons to show differences and similarities among 

- . the data. For example, after tabulating the enrollments for the different vocational education 

j. programs for the-previous year, they can be compared to enrollments for previous years. ^ 

^ .Increasing and decreasing enrollments in the programs may show tfiat a program should b^P 4 

. a ' expanded or dropped. Comparisons on ratings oHhe facilities over the years can supply m & 

r evidence for the need to remodel* ' 

Percentages are frequently used to indicate the number of responses to a question. 
Remember to report the percentage of responses as compared to the total number of responses 
to the question. To find ttlfe percentage dividsrtfre like responses to a question by the total 
numbefojf respondents. The following is an 4xample. 

** 

Eignty current inmates completed a questionnaire about their vocational 
education experiences. Twenty rated the counseling services as fair— 20+80 = 
* 25%. Thu&, 25 percent or one-fourth of the respondents indicated the counseling 

services were fair. 



t Using percentages lets you compare different sizes of groups. This is helpful when class 

sizes differ from year to ye^r. For example, It tast year sixty inmates returned the questionnaire . 
* and twent>N3ted the counseling services as fair, then 20-5-60 = 33 1/3 percent. Although the 

same number of inmates rated the serflce as fair this year as last, the percentage was higher last 

* • 

/ • 
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Sometimes it is helpful to know which answer or event ocpurs most often. Statisticians refer 
to this as tfre mpde or modal. The education leVel listed in taole 2 most often is eleven. Thus the 
mogal educational level is eleventh grade. The modal age of the inmates enrolled in the program 
is twenty-two. 



Table 2 

Example: Personal Data on Inmates Enrolled In Auto Mechanics 



^ £ Inmate 


Age 


tr 

Race 


Education Level 


A 


19 • 


Black 


" 8 v 


B 


22 , • 


Caucasian 


10 


C 


25 


Black 


11 


D , 


26 


Black v 


12 


E 


21 


Hispanic 


. 11. 


F 


19 


Caucasian 


12 


-G 


22 


Black , 


12 


* H 


28 


Hispanic 


9 


1 


20 


Caucasian 


11 


J' 


22 


Caucasian 


11 . 



The most common statistic is the average or mean. In table 2, for example, to find the 
averse add the nwhbers^or scores together and divide Jay the total nurtiber of items. Then add 
up all the ages and divide by 10. The total of 224 is divided by 10 which makes the average age 
slightly older than 22. Dividing the total of the educatipnal levels of 107 by 10'makes the average 
education level 10.7. 

Once the information has been collected and'analyzed, it must be converted to a form that * 
will be easily understpod by readers of the reports. According to Starr, displaying data 
information visually in tables, graphs, and so forth has several advantages. 

• Interest is created in reading the report. 

• Major points are highlighted. , ' w 
* • Information is. easier to understand and remember. 

• Information is easier to compare. 

• Information is easier to display, and use in presentations. 
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Tables 



Tables are used to compare or classify related numeral information. Information must be 



accurately compiled and arranged so that it can be easily read and interpreted. A table must 
have a title, and each row and column must havfe .Reading. Table 2 is an example of displaying 
fpfprmation in a way that is quickly and easily understood. 



Figures . 

/-* 

> Figures show the relationship between ideas or people or illustrates process. Examples of - 
figQres are organizational charts or layouts showing where the vocational education program is 
housed in a correctional facility. Figure 3 is an organizational chart used in Alabama in 1976. " * 



_ JIGUREJ 



• ORGANIZATIONAL CHART FOR ADMINISTRATION 
OF VOCATIONAL EDUCATION PROGRAMS 



State Board of Education 



" Superintendent of Edufcqfion 
Assistant Superintendent 



Director of Institute 



Business Manager & Coordinator of Instruction 



General Advisory Committee 



fcrafts & Occupational Instructors' 



I 



Students 



Craft or Occupational Advisory Committee 



I 

. I 
_ J 



Line Charts 



Line charts are effective in showing trends across time. For" example, a grid with yeaft down 
the left and'numbers of enrollments apross the* bottom is a flj^od way to show enrollment over 
the years. Figure 4 is an example of an effective way to show information. 
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Bar Graphs 

# * 
One of the most frequently used graphs is the bar graph. Since the size of the bar is 
proportional to the numbers represented, it is easy to compare sets of information. The bar 
graph is easy to construct and can be used effetf iveJy for comparing a large number of items. As 
figures 5 and 6 show, bar graphs can be drawn vertically or horizontally. 
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FIGURE 5 

BAR GRAPH: ENROLLMENT IN VOCATIONAL EDUCATION PROGRAMS 
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FIGURE 6 

BAR GRAPH: ENROLLMENT BY VOCATIONAL EDUCATION PROGRAM AREAS 
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Pie Charts 

Pie charts display how the "pie" of resources or time is divided. Each piece represents a 
percentage of the whole pie. Since the pie equals 360 degrees, each piece represents a 
percentage of 360 degrees. Thus, 50 percent is 180 degrees, 10 percent is 36 degrees, and so 
forth. To construct the pie chart, start at the 12 o'clock spot and mark off the largest section to 
the right. Continue clockwise in order of size.- Figures 7 and 8 are examples of pie pharts that 
indicate budget allocations and types of certificates awarded. 
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PIE CHART EDUCATION CERTIFICATES AWARDED 




Number Awarded 

GED 179 
Vocational 1 1 1 
High School 86 
College 52 



Total 

. \ 



428 
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Smithville continued. 



After the findings from the self-study and program review were tabulated and analyzed, the 
evaluation committee organized the information into a final report. The report identified several 
strong points along with recommendations for improvements. The report was sent to the warden 
who circu lated cb pies to the vocational edu cati on teache rs and administrators. The warden later 
met with them to discuss the evaluation results and recommendations. The education supervisor 
sent the report and a letter indicating what action would result from the evaluation findings to 
the review team members. 

The evaluation committee made several recommendations to improve the evaluation 
procedures that would continue. These recommendations included the following: 

• Establish a systematic working relationship with the parole boaW for inmate follow-up. 

• Encourage teachers to keep anecdotal records about inmates. 

• Have inmates and teachers complete an evaluation of the course at the end of each term. 

Pat and the evaluation committee were pleased with the way the evaluation was conducted. 
They understood th$ value of having an established system for continuous evaluation built into 
the vocational education p*99t%rr\. Pat realized that evaluation is useful for gathering information 
for planning, for program improvement, and for accountability, and is not merely an 
administrative requirement. One of the first unexpected side effects of the evaluation came when 
a local television station decided to do a news series on conditions in the state prison system^ 
During an interview with the reporter, the education supervisor referred to the evaluation report 
that highlighted the successes, as well as specific problems. The supervisor was able to discuss - 
the plans for changes that were underway. Use of the evaluation results .created a favorable 
impression of professionalism and accountability for the public monies spent for vocational 
education at Smithville. 
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The final step in evaluation is reporting the results in a useful way. Evaluations of vocational 
education programs in correctional institutions are conducted to provide information for a variety 
of, purposes that usually include decision making. Norton and McCaslin emphasized that the 
report should be written to address the audiences who will make decisions and who will be 
influenced by the decisions. The report should be written so that the various audiences are able 
to use different sections based on their need. You must know the intended audiences of your 
report in ordeV to present the information in the most usable ways. — — 



The following checklist 
report 



9 / 

^/will 



1 

ill help you decide the tyjjfe of audience for your evaluation 











Deciding upon Audiences for the Evaluation Report 
Please check yes or no. 


YES 


NO ' 




The evaluation report will be used by- 








1. State department staff 


□ 


□ ' 




2. Legislators 


□ 


□ 




3i Community leaders 1 '* 


□ 


□ 




4. Vocational education advisory committee members 


□ 


□ 


t 


5. Warden 




□ 




6. Education administrators (institution) 




□ 




Jfc Teachers, counselors, other staff 


□ 


□ 




8. Concerned citizens 

> 


□ 






9. ffublic media (TV, radio, newspapers) 


□ 


□ 




10.- Professional colleagues in correctional education 


□ 


□ 




11. Inrriates 


□ 


□ 
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Depending upon the audience, evaluation results can be reported in many f6rms. A brief 
report highlighting the major findings and recommendations may be presented orally to a group 
of legislators or community leaders, or presented as a news release to the public media. A 
detailed technical report may be prepared for state department staff, professional colleagues, or 
education administrators. The report may be written as an executive summary for legislators, or 
i written as A short report for teachers, the warden, and the vocational education advisory 
Veojnmittee. In general, those persons most closely associated with the program will need the 
\iost information. 

Whether spoken ot written, the report must have the essential information clearly presented. 
According to Adams and Walker, the report should include— 



• a cover page with title, date, author, and, if restricted, the names of persons authorized to 
read it; 

• an introduction describing the purpose of the evaluation, the questions to be answered, the 
intended audience, the objectives of the program, the correctional $§ftmg, the vocational 
education program, and the staff and inmates; ' « / t 

• procedures used to conduct the evaluation including sources of information, data collection 
procedures, instruments used, and data analysis procedures; 

• results of the evaluation, with tables and graphs describing the findings and conclusions] 
and N 

• recommendations that relate to the evaluation questions. 

The results section will be the heart of your report. It will describe how the desired outcomes 
have been met threugj^answers to the evaluation questions. The conclusions are based on the 
findings and tell what hasN^een or should be done. The recommendations, t stated in the 
beginning of the report, teUr who should do it and how. 

Prepare to write you/ report by organizing 4 the results on the following worksheet. For each 
evaluation question, lisf the findings and conclusions along with your recommendations. This 
provides an overview ol the information to be discussed jn the results and recdmmendations 
sections of your reportJ 



Write the report m a simple, clear, and objective style. Write in the third person and do not 
use personal refereiySes, such a^'l foand" or "we thought." Provide enough details but do not 
use unnecessary^ords or descriptions. Write out all abbreviations at least once. Have someone 
in the staff revfew the report before you have it typed for release. Have the final draft typed in an 
easy-to-read format Pages with wide margins and single-spaced, short paragraphs with a double 
space in between are considered best for readability. 

Present the report to your immediate supervisors and to the warden before it leaves the 
institution These administrators should decide who else should receive a copy. Encourage 
administrators and educatQrs to use the evaluation report for their decision making and as 
information in other reports they must submit. The program review team and evaluation staff 
members should receive A copy for their files. % 
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Worksheet 2: Preparing the Report 



Evaluation Questions 


Findings 


, Conclusions 


Recommendations * 


(What is the evaluation 


(What information was found 


(What has been or should 


(Who should do it and 


question?) 


to answer the question?) 


be done?*) 


how should they do it7 




• 








• 


* 






• 














• 

w 

/ 




- 


* 


*/ 




1 
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Usfe the following cheCtfTistAA .io determine if your recommendations are suitable for the 
evaluation: * c x r 



Determining the Suitability of Recommendations 



-please check.yes or no. * 
Are the recommendations— 

1 . Actually providing answers t6 eliTnmate the pjpblems identified? 



X 



. Not too time consuming to implement? 
t Short, concise, and still complete? 



4. Realistic for the institution? f n . 

$ - c 

5. 'Specific so they cannot be misinterpreted? 

6. Consistent with the desired outcomes of the vocational education program? 

7. Consistent with the philosophy of the institution?^ 

8- Reflecting the consensus of the evaluation team? 

9. Call for changes over a period of time rathe^than reorganizing the 
entire program? '* 



You have now completed the eight steps designed to guide you through evaluating ypur 
program. File your data carefully in preparation for the continuing evafluation and next report. 
You wilLfind that ybu can add the findingstyfrom future evaluations to the tables and graphs you - 
have already prepared. Remember, evaluation is a continuous process that must be updated on a 
regular bas^p. Be aJert for opportunities to add new ways to evaluate your program. 

* * - - * ^ » * 

The remaining sections of the handbook contain useful samples, descriptions of resources 
and r efina nces. * * 



PART III 
Bapkup Information 



sample materials 
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SAMPLE I: CHECKLIST FOR ACCREDITATION STANDARDS 

7 

In 1977, The National Center for Research in Vocational Education developed standards for 
vocational education* programs as part of the National Study of Vocational Education in 
Corrections project funded by the U.S. Office of Education. These standards were developed in . 
collaboration .wijh a panfci of experts in the fields of vocational education, corrections, and 
correctional education along with a project advisory committee. These standards have not ye* 
been adopted as part o? any agency's accreditation process but are being considered by sevpral 
professional organisations. • 

Use this checklist to help you evaluate ypur vocational education program in relation to. " 
accreditation standards. It^ay als^be used to help ^ou 'prepare for the accreditation process. 



Check yes, no or somewhat and add commen^when appropriate 
Curriculum and Instruction y ;« 



1. 



This correctional institution has wriflwf descriptions for each of the vocational 
programs. The descriptions explain the value and meaning of the course to the 
student, describe the type of career for which the training prepares the student, 
t provide an indication of recent job demand^lata, and define the relationship of 
the program to other educational and trainiflg-re lasted activities within the 
institution. ^ 



YES. NO t SOMEWHAT 

■□ □ ' □ ' 
Comments: 



This correctional \r\s\\tutUN$has statements of expected student performance for 
• each vocational program* performance objectives for each vocational program tell 
both teachers and students what is expected otthem in the vocational progpam. 
Objectives also provide a list of-what performances wilt be measured at the end 
of training. ^ * 0 ' 



YES- NO SOMEWHAT, 

d '□ or 

Comments: 



In this correctional institution, performance objectives are reviewed regularly 
with input from staff, advisory committee^rmmbers, and students, and are kept 
current with expectations held by businesi andindu^try and the labor market 
in general. 



YES NO -SOMEWHAT 
□ □ □ 
Comments: 



£ 4. This correctional institution has and observes a set of written criteria fty admission 
*r to each vocational pjfgram by which student eligibility for vocational programs 

is evaluated. They should be flexible to allow decisions to be made on the basis of 
individual student need, motivation,' and desire for participation in vocatjf 
education programs. Criteria for student ^participation in vocationa^edcfcation 
programs are a functioning part of the institution's intake and classification process. 



v YES NO SOMEWHAT 
■ □ □ □ 
Comments: 
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5. This correctional institution has 6n file for each of its vocational education pro- 
grams written comprehensive courses of study that include suggested teaching 
methods and procedures, and equipment, facilities, and supply resource fists. 



YES NO SOMEWHAT 
□ □ □ 
Comments: 



6. I n this correctional institution, the written courses of study are reviewed and 
revised as ftecessary to ensure their cof rectness and the timeliness of course 
content for each occupational area for which training is > provioyed. 



YES NO SOMEWHAT 

•an a 

Comments: 



This correctional institution has easily accessible learning resources such as~ 
textbooks, manuals, handouts^ booklets, tests, audiovisual, and other special 
materials necessary for effective and efficient instruction in each vocational 
course. 



YES NO SOMEWHAT 
□ □ ■ ' □ 
Comments; 



8. In this correctional institution, students have time to use the learning resources 
and are taught how to use the materials and equipment. 



YES NO SOMEWHAT 
□ □ □ 
Comments: 



Students 



This correctional institutibn has. an ongoing orientation program to acquaint ' 
students wifh the total vocational and academic educational programs. . 




Thiscorrectfonal institution has a guidance, counseling, and placement pp 
to test, evaluate, and counsel students in order to place them in vocational 
programs. The accurate assessment of students' needs prior^krplacing them in 
vooational classes better assures their success and ach^verrient iQ^those classes. 
While it rs desirable to havelhese services provided by the institirfton staff, it • 
is possible to contract for Jpfeir provision by another agency or school 



YES NO SOMEWHAT 

□ □ V □ 
Comments: 

Y§^4*0 ' SOMEWHAT 

□ \g .□ 

Comments: 



3. This correctionaOnstitudon maintains a student record system and educational 
files that are open to staff and to student review, subject to state and/or 
federal privacy laws. Open records promote accurate and fair/information 
reporting that facilitates better rapport between those reporting and those 
reported on. At the time of release from incarceration, a student's educational 
records are available to the student, prospective employers, and free-world 
school personnel. 



YES NO SOMEWHAT 

' □ □ ' □ 
Comments: , f 
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no 



r 



This correctional institution provides students the opportunity for practical 
application of skiffs acquired through vocational training during the rernainder 
of the student's stay in the institution* 



YES NO SOMEWHAT 
□ □ □ 
Comments: 



$ 5. This correctional institution has a studentevaluation program to test thoroughly 
and fairly a student's learning progress and to certify the attainment of com- 
petencies and/or skills necessary to various on-the-job activities. Evaluation 
includes pa per-and- pencil tests and practical application of skills learned to 
complete real job tasks. Students are aware of and actively involved in both the 
development of evaluation criteria and in the process of evaluation. 



YES NO SOMEWHAT 

□ a ' □ • 

Comments: 



6. This correctional institution provides students an opportunity to enter and 
complete sueh programs as may Had to appropriatedicensing and credential ng 
once training is completed and competencies afe certified through thejnsti- 
tution by the appropriate agency or group. When licensing is not necessary to 
a student's ability to obtain a job, completion of training programs is recog- 
nized by some type of diploma or certificate that also certifies the skills 
attained. Such certification is.recbgnized by schools and business and industry 
j in the free world." 



YES NO SOMEWHAT 
□ □ , □ 
Comments: 



This correctional institution has guidance, counseling, and job placement 
programs that $ eve I op jobs, make jot* available to ex-offenders, counsel 
students, and assist them in securing jobs appropriate to their job skills: These 
services are performed by trained instructional personnel or by personnel 
from another agency or school. 



8. In this correctional institution, a strong emphasis of the guidance, counseling, 
and placement programs is on the development of jobs within the community 
suitable for ex-students. ^ 



YES • NO SOMEWHAT 
□ □ □ 
Comments: 



YES NO SOMEWHAT 

□ □ 
Comments: 



This correctional institution has a comprehensive follow-up of graduates 
program to determine the relevance and success- of the vocational education 
activities and job placement services. 



YES NO SOMEWHAT 

□ □ □ . 

Comments: 



10. This correctional institution has a plan to make credits for vocational education 
earned here or at other institutions transferable to educational institutions in 
the community, such as community or junior colleges, area vocational schools, 
* colleges or universities. Students who do not have the opportunity'to complete 
■ a vocational educatiorvprogram prior to release from a correctional institution 
are permitted transfer* credits to a community program for completion. f 



*ES NO ; SOMEWHAT 

□ □ H 

Comments: 
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Stiff 



1. This correctional institution has a written plan for selection of vocational- 
administrators and faculty that is available to both employers and employees. 
The criteria are regarded as guidelines for staff selection. They are flexible to 
accountlfor individual differences in prospective employees and job position 
requirements. 



YES NO SOMEWHAT 



□ □ 
Comments: 



□ 



This correctional institution has a published salary schedule and fringe benefits 0 yes NO 

program for vocational administrators and faculty that includes a plan for | — | 
evaluation and promotion. Staff members are kept informed about salary scales, 

performance evaluation, and promotion criteria. Comments: 



SOMEWHAT 



3.*^This correctional institution has a written professional growth plan to provide 
for upgrading of occupational competencies of administrat6rs, teachers, coun- 
selors, and other staff through inservice activities, on-the-job experiences, 1 
participation in related professional organizations, and, additional college training. 



YES NO SOMEWHAT 
□ □ □ 
Comments: 



In this correctional institution, released time and compensation for inservice 
education and for participation in professional organization meetings are used 
to motivate staff to engage in profession^y^wth^cAvitjfs. 



7 
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YES NO SOMEWHAT 

□ □ ' □ 
Comments: 



5. This correctional institution has a plan to keep teachers, placement officers, 
. . .and counselors up-to-date in business and .industry -activities, and technology. 



YES NO SOMEWHAT 
Comments: 



This correctional institution has an evaluation plan that determines the 
adequacy of professional preparation, performance, and growth of each voca- 
tional education staff member. The existence Off an evaluation plarr keeps staff 

t members aware of the expectations of the institution regarding their performance. 

f 



YES . NO SOMEWHAT 
□ □ □ 
Comments: 



Organization and Adm initiation 

1. This correctional institution has a current and readily available written statement 
that describes the institution>wc8&o/i4/ education philosophy, programs, and 
ancillary services provided for inmates* This statement describes the relationship 
between the vocational education program and other functions and department/ 
areas of the institution. " » 



/ 



YES NO SOMEWHAT 
□ □ □ 
Comments: 
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This correctional institution has a general advisory committee for vocational 
education to advise the staff in establishing the philosophy, policies, and proce- 
dures for vocational education program operations. The committee is composed 
of people from the local business, industry, education, government, religious, and 
social communities. The committee members have the experience and ability to 
provide valuable and timely input to guide the vocational education program 
efforts of the institution. The advisory committee serves the vocational education 
program by providing liaison with the business community and by enhancing job 
development and placement efforts on behalf of the students. 



Y X ES NO SOMEWHAT 
□ □ □ 
Comments: 



This correctional institution uses vocational program trade and craft advisory 
committees to enhance vocational education programs for the purposes of evalu- 
ation, community relations, and curriculum development and revision. These 
committees serve in an advisory capacity to individual vocational courses or 
occupational areas within the entire institution vocational program. They pro- 
vide valuable information on trends in the field; input to curriculum updating; 
assistance in student placement; good public relations with the business and 
industry community in the free world; and, in some cases, assistance in student 
follow-up efforts. 



YES NO SOMEWHAT 
□ □ □ 
Comments: 



This correctional institution has a set of written policies and procedures for the 
* administration and operation of vocational education. 



YES NO SOMEWHAT 
□ □ □ 
Comments: 



In this correctional institution, an annual review of policies and procedures is 
conducted to provide a good evaluation of the extent to whtch.goals and 
objectives reflecting the vocational program philosophy are being addressed via 
those policies and procedures. 



YES NO SOMEWHAT 
□ ' □ □ 
Comments: 



This correctional instlution has properly qualified and Vor certified h staff 
(vocational education^administrators, supervisors) and necessary support 
personnel to operate the vocational education program efficiently and 
effectively. 



YES NO SOMEWHAT 
Comments: * * 



This correctional, institution has a plan for determining appr opriat e vocational 
education teaching load consistent with the characteristics and demands of 
the program being taught, the characteristics of the students, the nature of 
the facilities, and the needs of the teachers to r^fcn instructional time. Perform- 
ance objectives, characteristics of the occupational area, students, skills required 
by the job, and physical facilities are examined to determine how much instruc- 
tional and n on instructional lesson preparation time must be allocated to each 
program and how many students the program can accommodate. 



YES ~N0 SOMEWHAT 

□ □ a 

Comments: 
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8. This correctional institution has written financial policies and procedures that 
provide for stable program budgeting to supply resources necessary to meet 
vocational education objectives. * 



YES NO SOMEWHAT 
□ □ □ 
Comments: 



This correctional institution has written community relations plans for its 
vocational education program. 



YES NO SOMEWHAT 
□ □ □ 
Comments: 



TO. 



This correctional institution has a written plan for continuous planning, 
"research, and development activities dealing with vocational education program 
operations, policies, procedures,*curriculum, facilities, staff, equipment, and 
budget£hort- and long-range planning activities' are broad enough in scope to 
include the total vocational education program from curriculum development 
and revision to facilities use and maintenance. Planning, research, and develop- 
ment are either conducted by internal staff and/or by personnel from an outside 
agency who are qualified to perform such functions. 



YES NO 

□ □ 

Comments: 



SOMEWHAT 
□ 



11. In this correctional institution, the results of research efforts are used to alter YES NO SOMEWhAT 

and improve educational activities. u □ □ □ * 

Comments: • V 



12. This correctional institution has a written plan for continuous collection of 
evaluation data about vocational programs, operations, policies, procedures, 
curriculum, facilities, students, st^ffT^uipment, ^nd budget. Evaluation is 
conducted by persons from within or outside the system. 



YES NO SOMEWHAT 
□ □ □ 
pomments: 



13. This correctional institution has a written plan to identify and attempt to 
eliminate discrimination on the basis of race, color, creed, sex, and/or condi- 
tion of handicap in staff selection and assignments, student selection, and 
planning and development of curriculum and instruction. 



YES NO SOMEWHAT 
□ □ y □ 
Comments: 



Physical Plan, Equipment, and Supplies 



1. This correctional institution has a documented plan for the operation and use 
of vocational education program facilities, equipment, and supplies including 
use of manuals and emergency procedures. 



YES NO SOMEWHAT 
□ □ □ 
Comments: 
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2. This correctional institution has a plan for preventive maintenance and house- 
keeping activities related to all vocational facilities, equipment, and supplies. 



YES NO SOMEWHAT 
□ □ □ 
Comments: 



This correctional institution has a plan for short- and long-range development 
of new facilities; acquisition of new equipment and supplies; and modification 
of existing facilities and equipment for vocational education programs. The 
plans include the need for and development of new facilities; the improvement 
of existing facilities, equipment, and supplies to support existing and planfned- 
for vocational programs; the acquisition of new equipment; and the planned 
replacement of equipment when it becomes worn out or obsolete. 



YES NO 
□ □ 
ments: 



SOMEWHAT 
□ 



4. In this correctional institution, the vocational education program's safety and 
health conditions meet local, state, and national standards. These standards are 
used and adhered to by this institution to ensure safe and healthful working 
and learning conditions for staff and students. 



YES NO SOMEWHAT 

□ □ Q 
Comments: 
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SAMPLE 2: FORMER STUDENT QUESTIONNAIRE 



The following questionnaire was adapted from Evaluation of Learning: The He^rt of 
Instruction, Ohio Department of Education, and is keyed for computer tabulations. 



* 



DIRECTlOlStt^ 

PLEASE CIRCLE THE NUMBER AFTER THE APPROPRIATE ANSWER TO THE QUESTIONS. 



\ * 

1, What is your currant tmploymtnt status? 

I am am ploy ad full tin* (30 or mora hours/waak) . 

I am ampJoytd part tima (lass than 30 
hourc/waak) 

I am unampJoy td but looking for work 

I am unamoloyad and not looking for work 

I am in tha military sarvica 

If you art ampioyad full tima 90 to quastion 3 

2. If you art prasantty unamoloyad, tadtcata tha 
appropriata reason. » 

I am attanding school --Ml] 

I am a houtewift or about to bt fnarriad [2] 
1 ravt a handicap which pravtnts mt from working [3] 
I am not mtarastad m a job 



U) 
I3J 



I only wanted to work part t»mt (lass than> 
hours/waak) 



Othar (spacifyL. 



X Wart you satisfied with tha training you raeahrad 
tn this program? 

Yas |1) No 12) 

4. Would you racommond this program to othars? 
Yas (lj No (2) 

5. How many fun tima Jobs (30 or>ora hours par 
waak) hava you hs4d sinos you loft this program? 

t^arn ampioyad in tha sama job I had prior to 
laaving tha program . K 



(4) 
16) 



Ona full tima job 

Two full tima jobs 

Thraa or mora full time job* 

IP YOU ARE UNEMPLOYED. PLEASE STOP HERE. 



in 
121 

13] 
14) 



S. How dosafy dots your prasant job ralata to tha 
training you rtctrrtd in tha program? (chack only 
ona box) 

I am ampioyad in tht occupation for which I was 
trainad by this program 



I am ampioyad in a rtiatad occupation .*. 



I am ampioyad in a compittaty diffarant 
occupation 



Prasant occupation. 



Othar (sppcrfy). 



in 

12] 
13] 



7. Did this program halp to pr apart you for your 
prasant job? 

Yas (1) No (2] 

t. How did you gat your first full tin* job aftar 
laavtng this program? (chack most appropriate 
box) 



I got tht job mysarf 

My family or fr lands halpad ma gif tha job 



Tha job piactmtnt sarvicas providad by tha school 
halpad ma gtt tht job , 

Tha statt amploymant sarvict halpad ma oat tht 

jot : 



in 

12] 

r3i 
1*1 

(51 



i. What it your monthly salary? 



Lass than $100.00 
$201.00 to $200 00 
$20k00 to $300.00 
$301.00 to $400.00 




(1] $401.00 

|2] $501\00 
13] 
14] 



10. What spacific things about this program haWyou 
found to ba tha most usaful to you indoor pra 
*»? 



11. Baaad on your txpsrianeat, what* suggastiom do 
you havt for improving this program? 



ADDITIONAL COMMENT*' MAY BE WRITTEN ON THE 
REVERSE SIDE. 
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SAMPLE 3: EMPLOYER SURVEY 

This survey was adapted from the state of-Maines Vocational/Technical Student Follow-up 
pystem. 



PLEASE RESPOND TO THE FOLLOWING QUEST k)NS BY CIRCLING THE APPROPRIATE NUMBER. 

/ 

I. In comparison with other workers of about the same age, and who have beenjon the same job or a similar one, 
for approximately the same length of time, how would you rate tKis employee on: 

Excel 

• lent Good P< 

• 1. The quality of employee's worfc 1 

2. " The quantity of employee's work \ 1 ^2 

3. The degree to> which the employee possesses specific job-related » 

knowledge important to success on this job 1 , 2 3 4 

4. The degree to which tfie employee is able to operate the 

equipment and apparatus used on the job 1 2 3 4 

5. The degree to»which the employee possesses adequate reading, 

verbal, andcomputational skills \ . . . 1 ^ 2 3 4 

6. The degree to which the employee possesses specific job- re I a ted 

» knowledge necessary to advance in this field 1 2 3 4 




II. In comparison with other workers of about the same age, and who have been on the same job or a similar one, 
for approximately the same length of time, how would you rate this employee on: 

1. ft illingness to accept responsibility , 

2. Judgment, ability to make decisions, plan, organize 

3. Ability to work without supervision T 

4. Willingness to learn and improve . ^ 

5. Cooperation with co-workers \ ........ . 

6. Cooperation with management 

7. Compliance with company policies, rules, and practices 

8. Work attendance/punctuality . . 

9. Cusfomer/client relations . . . . 

til. Did employee require any on-the-job train 

If so, approximate number of hours: Hours 



1 . 2 


3 4 


1 2 


3 4 


1 2 


3 4 


1 * 2 


3 4 


1 2 


3 4 


1 2 ■ 


3 4 4 


1 2 


3 4 


1 2 


3 4, 


1 2 


3 4 


Yes □ 


NoD 



IV, Did the fact that this person graduated from a vacation a (/technical program influence * 

. your decision to hire him/her? * Yes □ No □ 



c 



SAMPLE 4: COVER LETTER TO PROGRAM REVIEW TEAM MEMBERS 



* 

\ 

^Letterhead 



Dear Mr. Smith: - 

Thank you for your positive response regarding the Program Review Team 
Evaluation of the Smithville Vocational Education Program. Your 
services' will contribute to 'the occupational preparation offered at this 
Institution. Enclosed* is a packet of information concerning the role 
and duties of a Program Review Team Member and also a description of the 
program that you will be helping to evaluate. Please review this 
information prior to the visit. 

As previously discussed, the meeting Is scheduled for Tuesday, May 1, 
1982 at 9:30 a.m. Your interest in vocational education and your par- 
ticipation as'a feam member are appreciated. Your ..involvement in all 
activities is encouraged. It is especially important that each team 
member stays for the entire session and has his/her contributions 
included in the summary report. ' 

9 

If you have any questions after examining the materials, please tele- 
phone us at 614-492-8888. We are looking forward to working wit+i^ou. 

Cordially, 



Pat 'Howell • '[ 

Smithville Vocational Education Supervisor 
PH/slw / 



Enclosures 



J 



J 



SAMPLE 5: INSTRUCTIONS FOR PROGRAM REVJEW TEAM 

This set of instructionswas, adapted from the Local Educational Assessment Program, 
Georgia Department of Education. 

9 

A proqjam evaluation involves the visitation, observation, and analysis of an instructional 
program by a team of individuals with expertise in the are^ and with the purpose of providing 
suggestions for improvement. 

Previsit Duties 

1. ^ Study the curriculum outline of the program to be evaluated. 

2. Compare the curriculum being taught with actual needs of business and industry. 

3. Examine the Program Review Team Evaluation forms that will be uspd as a guide during the • 
actual team visit. \ s 

4. Make an effort to contact all faculty personnel and as many students as possible. 

5. Record comments and observations on th§ Program Review Team Evaluation Form. 

6 Meet with the Institutional Representative fQrjfceSummary Conference to discuss the visit, 
review and assess their findings, and to ffSSke recommendations. 

7. Submit individual Program Review Team Evaluation Reports to the Institutional 
Representative and clarify any Ambiguous statements or responses. 



\ 
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EXAMPLE 6: PROGRAM REVIEW TEAM EVALUATION REPORT 



This example was adapted from the Local. Educational Assessment Program, Georgia 
Department of Education. 



m v 

Program: 
Team Member: 
Business Affiliation: 



Date^ 

0 



The Program Review Team Evaluations members examine and make recommendations in the 
following areas: 



1. Physical Facilities 

2. Instructional Program 



3. Equipment ^ 

4. Supplies 

5. Instructional Materials M 

6. Upgrade Training 

Each Program Review Team Member should visit the area of the program being evaluated; 
observe the physical facilities, equipment, supplies, etc.; interview all instructors and random 
sample students; examine program goals and objectives, handouts, etc.; and ask for any needed 
interpretations and clarifications, While in the program ar$a, the team member should record 
responses and observations on the Program Review Team Evaluation Report. The ! 
recommendations may/6e written then or during the Summary Conference session. 



ERJC 
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sample 7: Evacuation form 



This evaluation Tdrm was acjapted from the "Locat Educational Assessment Program." 
Georgia Department of Education. % ' * * 



9 



1. Physical Facilities - To haveVffective instruction, the physical facilities must meet the needsof tRe program 
^ and student$;,Please rate the following:,-- 1 . * 

aluation Scale- 

3 points: Ei ccetnit - Needs no imoffovement 

2 points: A Jeaciate - Needs slight improvement 

1 point: 'Inadequate —Needs definite improvement 

r 0 points: N6t Applicable*- Does not apply ^ 



a. The^size of the atassroom(s) 

b. The size of the laboratory area(s) 

c. * The righting in^e dassroom(s) 

d. The^kjhting in the laboratory' areajs) * 

e. Gas supply, if needed fo§ instructfonal operation 

f. Water supply, if needed for instructional operation r 

g. Electrical service* avai labJe ^ . 

h. t The nofse level of the department 

i. Safety conditions rf> the classroomfsf " ^ 
. j. Safety cooditions In the laboratory area(s) k 

J{fc Housekeeping-neatness and orgartization • 

* I.. Proper storage facilities ', / 

n> Walking/working space available for free and , ° 
unobstructed movement in the area(s) . 



» y n 

3 pomts < 


r 

2 points 


1 point ■ 


0 points 






» 




— * 
































T — ' 

































































Comments: (Please identify" classroom by giving room numbei.) 



t 



1 



~7 



cy\A4/wrL£r. nffujir iu/ neeps rive aoamonai ^ecztjcai outlets. 

'• f • . . • • 


* * 

% 1 








■ ■ •/ ' • - 


— - ■ " : v ■ «\ ■ 


/ 9.- 

— ^ 1— 
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t. instructional Program 

* (a) • Are program goals and objectives clearly stated]? If no, identify those needing change. YES □ 

•_Jw ■ • . — ■ 



NO □ 



(b) Are-provisions made to eliminate sex or racial bias in this program? If.no, identify YES LJ 

, needed changes. * • NO d 



(c) Are program goals and objectives consistent with occupational standards? If no, identify , V|S d 
those needing change. NO □ 

— — — — — - 



\ 



: 2 : no D 



(d) Are program goals and objectives in printed form? . YES 



(e) Are student teaming experiences consistent with occupational practices and standards? * ' YES □ • 
If no, identify thlbse needing change, f m % NO □ 



(f) Are the occupational techniques being taught consistent with occupational practices • w YES □ 

and standards? If no, identify those needingxhange. • ' NO □ 



tg) Are provisions made within the program to-tfork with special needs such as disadvantaged YES □ 

v and handicapped students? ff no, identify needed modifications. NO %□ 
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3. Equipment *• 

(a) Is the equipment in this program modern and current according to the standards of 
business and industry? If no, identify that which is'put of date. > 

: 



YES □ 
NO □ 



i / ; 



\ (b) Is sufficient equipment available for student use? if no, identify specific equipment 
where additional quantities are needed for student use. 



VJES □ 
NO □ 



(c) Is the equipment in this program in proper operating condition? If no, identify that ^ YE S I 

* which needs repair. , , NO O 



(d) Is an adequate preventive-maintenance plan used with the equipment in this program? 
If no, suggest ways for improvement. 



YES □ 
NO □ 



(e) Js all the equipment in this program safe for student use 7 ti no, identify unsafe 
equipment, 

— — Y — - — : — " — - 



ves E3 

NO □ 



(f) 'Based on your, observation, 4 r e students practicing proper safety prdcedures? If no, 
'identify improper safety procedures. V 



(g) Is equipment available sufficient to mee^new and emerging trends in business and 
industry? If no, identify new e$uipmem>needed. ^ 



i- 
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YES □ 
NO □ 



I 

YES □ 
.NO □ 
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- • - * 

• \ 

4. Supplies 

\ 

(a) Are currently needed supplies on hand or readily accessible? If no, identify 



)se needed. 



YES □ 
NO □ 



(b) b Are supplies insufficient quantity to meet student heeds? If no, identify supplies where 
greater quantity needs to be maintained. 



YES □ 
NO □ 



" ~ — " : : ' 

(c) H the quality of supplies consistent with those used in the occupational setting? If no, 
identify supplies where the quality is not cQn^stent with those used in the occupational 
setting. y ' . , 



YES □ 
NO □ 



[d) . Are supplies efficiently expended so as to minimize waste? If rp, pleare give suggestions 
for improvement.' 

\ . * . 

■■ f *- 



YES □ 
NO □ 



5, instructional Ma'toriafy 



/ 



(a) Is there sufficient quantity of printed instructional -materials (such as supplemental 
reading lists, related practice exercises, handouts, etc.) available? If no, identify those 
needed., 



(b) Are the printed materials up to date? If no, identify those that are outdated. 



YES □ 
NO D 



YES □ 
NO □ 



v (c) , Is there a sufficient quantity of audio-visual instructional materials available? If no, 
identify those needed. 



-t r~ 



YES ,□ 
NO 



• r ...... / 

(d) Are the audio-visuals up to date? If no, identify those that heed upda^ng. 

S 1 ' 5 fep^* " 



YES □ 
NO 



*(e) Is there sqfficient quantity of reference materials available 'Tor ,student use? If no, 
identify those that are needed 



YEfe .□ 
NO □ 



(f) Is sex or racial bias eliminate</in the .instructional materials used in this program? 
If no, identify specific materials that are slanted toward a particular sex or race. 



Y,ES .□ 
NO □ 



\ 



\ 
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EXAMPLE 8: CRITERIA FOR EVALUATION OF' 
VOCATIONAL £DUCATIQN PROGRAM 

The fqjlowing^uesttonnai^e can be used as part of a self-study project or by the program 
review team to evaluate your vocational education prograrrvjt was adapted from materials used 
by the Minnesota Department of Corrections. I * 



PART I: PHILOSOPHY 

1. What is the philosophy of your vocational program? 



2. Has the philosophy been discussed with— 

a. Administrators^ * 

b. Supervisors (local)? ' • 1 
a Supervisory (state) ?|| ~ 

d. Instructors? * 

e. Students? « 

3. ^ the philosophy of the vocational program compatible Vv^h thafof the institution* (Explain) 



yes 


NO. 


□ 


□ 


tD 


P 


□ 




□ . 


a 


d 


□ 



» t 














1 






' -4- 



PART II: PROGRAM AND COURSE OBJECTIVES " 

Program Objectives - 

1. The vocational education course*should prepare individuals for living and employment by developing— 

a. Attitudes accepted by the employer and society 

b. Entry- level skills . v . • 

c. Work-related skills ' 

d. Communication skills related to work and living 

e. Human relation skills . 
f< Family and/or community responsibility 

g. Dignity in quality workmanship 

* ' \ * «• ♦ 

2. Tr^firfstudents to«a level of acceptable quality to Vneet employer needs 



£ Provide an effective and practical plan for post-training follow-up 

4. Initiate an exact plan to obtain community support of the vocational program* ^ 



□ 




• n' 


□ 


•a 


a 




a 


□ 


□ 


□ 


a 


□ 


ND 


a 




□ 


D. 


□ 



Courst Objectives 

1. To provide an atmosphere conducive to learning 

2. To assist each individual in recognizing a need for learning and behavior changes 

3. To hqjp individuals understand themselves and their environment 

4. To develop within each individual a sense of self -worth * 

5. To provide learning experiences related to living and earning a living 

PARTEM: ADVISORY COMMITTEES 

♦ • 

1. Consist of statewide industrial personnel— one group to advise all institutions 

2. Confirm or suggest improvements to course contents * * • 

3. Advise on education level and qualif i canons for— 

a. Training course entry ^' 

» *» 

^b. Job entry 

4> Advise on-job a vai lability - * . ^ 

5. Advise on facilities and equipment 

(Questions to be developed after committee is formed) ^ 
PART'lV:' COURSE OUTLINE , 

1. Is there a wor\iftg^>utline available fpr the course 7 . * 

2. Is the outline made current periodically as changes in the field occur? 

3. Is the working outline structured proper fy? 

C , 

4. Are instructional aidsjdentified specifically and numbered or lettered for efficient use? * 

•* " 

5. Is there a time element listed for— 
a. Each item? 

Each unit (subto|pl) ? y 
c. Course (total)?* \ ' 

6. Is the put line, in consistent use? * * 

7. Are listed references up-to-date? 

8. Ka$ the course content been confirmed by the advisory committee? 

Part v: Laboratory or industrial facility 

• r 

1. For practical work issignments. is there ample space? 

2. - Are all work stations visible to the instructor? * 

• ' f . ' 

3. Is there sufficient classroom space available for, teaching the related and theory content 

of the course? . ' 

.. • 

4. Does the facility lend itself to being flexible for teaching from various parts df the 
+ course outline? , 1 * 

*' 

5. Are storage areas adequate and orderly?* . 



YES NO 

□ □ 

□ □ 

□ □ 

□ □ 

□ .a 

□ • □ • 
a □' 

— □ □ 

i 

' * i 

. . I 

d 'Ep 

, ' a a 

□ □ 

□ □ 

a □ 

□ □ , 
% □ □ 

' □ , □ 

■ ' ' □ □ " 
.□ □ 

■ a D .. 

□ a 

□ - a 
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necessary, do students have proper project storage facilities? 
Are utilities sufficient? ^ 

Do safety features exist where required? 

a. /Electrical? (restricted and supervised use) 

b. Firefr (ample extinguishers) 
4 c. Student protection? (glasses, gloves, etc.) 

d. Equipment? (guards, etc,) 

9. Are the acoustics and ventilation adequate? 

10. Where required, are dust jpllection and exhaust systems adequate? 

1 1. Is there adequate lighting in— " ? 
• a. Laboratory or industrial facility? 
r b. Classroom? • 

c. Storage area? 

12. Are major a.nd minor maintenance repairs done as soorfas possible? 
" ^13. Is the Jaboratory-classroom location convenient to related facilities? 

t4. Do the faboratories contain an industrial color coding system? 
f ' 15. Are good housekeepinf practices evident? • t 

■ : ■ * / 

PART VI: TOOLS ANp EQUIPMENT / 

■1. Does each Work station have the necessary hand and/or power tool/for use in 
completing assignments? " " 

2. Has consideration been given to the arrangement of tools and eJjipment to 



of tools and etxnpmer 



1^ 



* provide opporti^y for etficient use?, | A 

1 Is power equipment properly located to promote safety? 
C 4. Are tools and equipment properly maintained? , 



5. Are tools and equipment used an accurate representation of. those used in the 
world of work for that vocation? . 

( 6 N Is therein efficient procedure developed for repairs and/or replacement of 
defective tools and equipment? 

7. Are all necessary visual aids available? , m - 

8. Do the faci lit ies>nclude a visuaf aids laboratory? 

9. Are there ample bulletin boards in good condition? v 

KrJ VII: MATERIALS AND SUPPLIES 

1 • , •* ■ m «■ ' 

1. Are there sufficient funds budgeted for ordering and maintaining a sufficient 
: slock of materials and supplies used for instructional purposes? 

i» * 

2. Is the procedure for odenng replacement materials and supplies efficint? t % 

V • * - 

J. Do materials and supplies used to complete practical work assignments correspond - * ^ 

to those used in the world of worfc? * . . ' 1 ^ 
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YES 


NO 


□ 


n 
L — '» 






□ 


a 


□ 


' □ 


i — j 


1 1 


□ 


□ 


□ 


□ 


p. 


Dj 


□ 


□ 


□ 


□ 


□ 


□ 


□ 




□' 


□ 


a 


□" 


□ 


□ 


i — i 


. LJ 


□ 




□ 


□ 


( — l 
i 1 




1 — 1 


1 — j 
1 1 


□' 


□ 


□ 


.a 


□ 


□ 


□ 




l — l' 
I_J 


i — i 


□ 


□ 


P 


□ 


□ 


p 
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PART VIII: INSTRUCTION * YES NO 

1. Are classroom furnishings conducive to the teaching-learning environment^ , □ □ 

2. Is there a teaching-learning atmosphere in the classroom environment? □ □ 

3. Are the grooming habits displayed by the instructor acceptable? 

4; Is an accepted technique of instruction followed with some finesse? For example— ^ 

a., Preparation □ □ 

b^-Motivation □ □ 

a Objectives < -„ \ □ □ 

d. Presen|tation 

e. Review 

f. Assignment 

g. Follow-up . LJ Lj 

5. Do* teaching-learning assignments follow in an orderly sequence? • d D 

6. Does each student thoroughly understand how to complete a given assignment? - Q O 

7. Are students permitted foj/vork aTa rate 6om par able to their ability? , □ □ 
(Assuming proper placement) 

8. When possible and practical, are students taken to a related industrial environment? . . " 

9. Do the entry-level skills being taught-learned meet the requirement$ of industry? . □ □ 
1Q. Do the students'display a sincere interest in their classroom-laboratory assignments? * Q D 

11. Is there a display of an acceptable attitude in and out of the teaching-learning environment? a Q □ 

12. Are instructors free toVnake student progress evaluation and to make recommendation* 

fo£ reassignment. of students? , ' — ' — ' 

13. Is instruction being offered at the level as indicated by the course outline? . □ □ 

14. Is there evidence of good student-teacher rapport? * Q D 

15. Is there specific effort put forth with a purpose of improving attitudes when necessary? C3 □ 

16. Are health problems corrected £s soon as possible? D 

17. Are job or practical work assignments prepared^ "handout" material? d d 

18. Does the dassroom or laboratory ha*fe definite organization? 

19. Is the average attention span for the group considered? J □ C3 

20. Are there standardized tests fi 
% a. Course entry? 



b. Course progress? 



tsforV 




□ □ 

□ □ 

□ □ 



c. Course" completion? * \ 

• \ ■ , 

$1. Does the training'period recommended provide ample time for learning entry-level skills^ □ □ 

fARTlX: ADMINISTRATION ^ 

1. 6o personnel from* industry^ t the local institutional program for evaluation purposes* / O □ 

2. Are instructors consulted by assignment, plaoement, or classification committees on ^ 

placement, of st^dinU in fheir classes? ^ y : I 1—1 

( 



3. Does the instructor have access to a visual aids laboratory (local or state)? 

4. Is the class within the recommended studenfrcount? 

5. Inhere an established program for retraining or for attending college-level courses for instructors? 

6. DoesUhe program have criteria for reassignment of,a student when necessary? 

^ Is there specific and accurate communication among members of a treatment team or 
. institutional classification comrriftee? , 

8. Are methods used for evaluating the same course, offered in one or more programs, uniform? 
• 9. is there some definite procedure for routing practical work problems? 

10. Are tests available for each course designed to determine level of entry into the training or 
progress achieved during training? 

1 1. Does the follow-up plan meet the needs of the indiyidual'for the required time period? 

12. Do the classroom academic instructors relate the subject matter they teach in the areas of 
computation and communication to the vocational education courses? 

13. Are there extracurricular activities organized to prombte vocational education In 
the institution? , , 

14. Is there a procedure for determining certificate expiratipn date available to the institution?, 

* 15. Are certificates of credit given only to those who have salable entry-level skills for a 

specific vocation* 
t 

16.- Is there definite support for apprenticeship trainmg and on-the-fob training w^here applicable? 
PART X: COMMENTS' \ 



YES NO 

□ □ 



□ 
□ 
□ 

□ 
□ 
□ 

□ 
□ 
□ 



□ 
□ 



□ 
□ 

□ 
□ 
□ 

□ 
□ 



□ □ 
a b 



□ 
□ 



The institutional educa; 
' vocational education p 
Those adm'tni strait or*, i 



Itmpr 

nroTvec 



program staff, particularly those concerned direqtiy with one or more aspects of the \ 
m, will be expected to b/come very familiar with the total conterfts of this set of standards, 
ed only indirectly, are requested to di/ect their attention to Part IX. ' _ 



COMMENTS BY INSTRUCTOR: 



COMMENTS BY INSTITUTIONAL EDUCATION SUPERVISOR: 



Signature 



COMMENTS BY SUPERINTENDENT OF INSTITUTION: 



Signature 



\ ■ 



r 
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resources 



The following selected materials can provide additional iftforrgation for evaluating vocational 
education programs in correctional institutions. 

• « 

• Materials Available from the National Center for Research in VocatiorW Education 

Reports on i National Study of Vocational Education in Corrections 

f - ■ 

VED 05 . Jones, Joan Simon. Vocational Education in Corrections: An Interrelation of $ 3.80 t 

Current Problems and Issues— National Study of Voca tional Education 

in Corrections— Technical Report No. 1. 1977, 53 pp^ Reports the results > 
n - ^ of computer-assisted and manual searches of the literature on history, issues/ 

and trends in vocational education in corrections. 
\. • ■ 

VEO06. <► National Center for Research in Vocational Education. Standards for Vocational $ 2.20 

^s^" ^ v Education Programs in Correctional Institutions: National Study of Vocational 

" Education in Corrections-Technical Report No. 2. 1977,17 pp. Presents 4 

standards in the areas of curriculum and instruction, organization and adrninis- ^ 
- tratiorf, -equipment and supplies. Standards were developed to help corrections Vf 
per£qnnel establish goals and develop guidelines for programs in vocational * % 
- ' jcainingT * 

VED 07 - Abram* Robert,*and Schroeder, Paul C. Vocational Education in Correction Inst 'h » $7.75 

i , • * tutions: Summary of a National Survey-National Study of Vocational 

Educationin Cor/ections-Technical Report No. 3. 1977,123 pp. Provides 
dat^oarfourteen areas in vocational programs in correctional institutions, 
including facilities, prograrrfgoals and Organizations, student characteristics, 
and costs. ^ f • j ^ ' 

VED 05-07 Available as a set « ' $11.00 

Other Resources * ^ * - 

IN 158 Hopkins, Charles 0. Data Sources foryocational Education Evaluation. ^79* 2$ $ iz.35 

pp. Identifies data needs at all school levels,. lists dall source informatiort, and 
i- outlines limitations on the use of data for evaluation. " 

" ■ - • y " ! M * : 

RD 182 Stevenson, William W. Vocational Education Evaluation: Problems; Alternatives, • *$ 4,50 

Recommendations. 1979, 61 pp. Suggests several possible approaches a state 
should consider in seeking methods for improvement of a vocational eiiucatiQn * 
* - program. 1 ♦ • 

Evaluation Handbooks for Vocational Educttion ♦ s — 1 ■ r 



$i(g 



RD J 71 ^ Franchak, Stephen J., and Spirer, Janet Eg Evaluation Handbook Volume I: * $13.00 
* guidelines and Practices for Fo/lpw-yp Studies of Former Vocational Students. 
0 1978, 230 pp. I ndudes specific techniques, legislation and requirements, 
_ development of forms anc^ instruments, guides for interpreting data, alterna- 
tives for reporting, and a bibliography/ ■ ' 
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RD 172 Franchak/Steph^n J., and Spirer, Janet E. Evaluation Handbook Volume It: ' $1A-25 

Guidelines and Practices for Follow-up Studies of Special Populations..A979 e 
• 273 pp. Examines context definitions, and strategies. Identifies current 
t problemsand issues along with existing practices that have proven to be 

fi successful Although former inmates are not specifically identified, many'of \ * ~ 

the sam^problems occur witf\ them as with other types of special pbpulations. 

RD 171 and Available as a^et m $25.00 
RD 172 * * , 



Career Education tfeasuremeAts Handbooks 



: to d/my« 



♦ The Career Education Measurerrfeot Series cdnsists of the five handbooks that are rheant to <Ltfmystify career 
education evaluation for the professional who is not an evaluation socialist. 

RD 165 Malak, Sharon; Spires Janet E.; and Belligrini-Land, Brenda. Assessing Experiential $ 7.75 

, Learning in Career Education. 1979, 124 pp. Focuses on how to evaluate 

* experiential education and provides checklists, resources, and other information 
to help in planning, fmplerneptfng, and reviewing assessment strategies. 

RD 166 McCaslirv, N.L.;'Gross, Charles J.; and Walker, J^V p - Career Education Measures:^ $17.00 
A Compendium of Evaluation Instruments. 1979, 354 pp. Contains over 200 
abstracts of instruments currency used by practitioners in career education. 

* *RQ 167 . l McCaslin, N.L., and Walker, Jerry P. % AGuide for Improving Locally Developed $3.80 
/ , Career Education Measures. 19Z9, 25* pp/Contains step-by-step process for r 

developing or modifying instruments. - ' — 

1 RD 168 Adams, Kay, and Walker, Jerry R. Imptovin^ ihe Accountability of Career Education , $ 6.75 
Programs: Evaluation Guidelines Checklists.- 1979, 1Cf3 pp- Shows hqwlmproved 
evaluation can help make career education programs fare better irr presenting 
evidence of accountability. • 

RD 169 ' Kester, Ralph J. Using Systematic Observation Techniques in Evaluating Career $4.50 
Education. 1979, 64 pp. Provides a series of techniques that can'be used to * 
r « show how the program is working. v - 

»"•*«- * 

RD 165-169 Available as a set $30.00 



r 



National Center materials are available on a c*st-recovery basis from: 

The National Center for Research in Vocational Education 
National Center Publications, Box E* . 
1960 Kenny] Road ' ^ 
Colymbus, Ohk? 43210 



When orderin g, ple ase uy order number and title. Orders of $10.00 or less must be prepaid, Majce check payable to 
the National Center for Research in Vocational Education. Prices listed ace in effect at the time of publication^nd 
include postage and handling. For discounts on quantity orders, contact the National Center. : 
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Materials Available from the American Associatic^for Vocational Instructional Materials' . g 

The following modules were developed by the National Center for Research in Vocational Education a/id are 
available from: 

American Association for Vocational Instructional Materials 
1 EngineeringXenter * , * 

University of Georgia * 

Athens, Georgia 30602 , . 1 

* 

Each module is a workbook designed for individualized learning. 

Module « . ' Prices 

Number _ - » ^ 

\ 

A-3 Report the Findings of a Community Survey. 1978, 64 pages** Designed to develop , ^ $ 3.75 

skills in analyzing survey data, reporting clearly and concisely the results Of that 
analysis and disseminating 'the findings. 

A-10 Conduct a Student Follow-up Study. 1978, 64 pages. Designed to acquaint the reader $ 3.75 

with follow-up procedures and develop^kills in planning and conducting a 
student »fol low-up study. 

A-11 Evaluate your Vocational Program. 1978, 56 pages. This module focuses orr locally $ 3.45 

directed and product oriented program evaluation. It is designed to develop 
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